Model / supervised · All operations

Classify by nearest neighbours

K-nearest neighbours.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

Shape of the result
a model or file and a table in, the table out with columns added
Level
several clauses of SQL, or a window

The DuckDB step

Classify by the three nearest labelled rows

SQL
CREATE TEMP TABLE result AS
WITH train(id, x1, x2, label) AS (VALUES (1, 0.1, 0.2, 'a'), (2, 0.3, 0.1, 'a'), (3, 0.2, 0.4, 'a'), (4, 4.0, 3.1, 'b'), (5, 4.2, 3.3, 'b'), (6, 3.9, 2.8, 'b')),
     new(nid, x1, x2) AS (VALUES (10, 0.2, 0.2), (11, 4.1, 3.0)),
     ranked AS (SELECT n.nid, t.label, row_number() OVER (PARTITION BY n.nid ORDER BY list_distance([n.x1, n.x2], [t.x1, t.x2])) AS rk FROM new n CROSS JOIN train t)
SELECT nid, mode(label) AS predicted FROM ranked WHERE rk <= 3 GROUP BY nid ORDER BY nid;
SELECT * FROM result;
SELECT CASE WHEN ((SELECT map_from_entries(list((nid, predicted) ORDER BY nid)) = MAP {10: 'a', 11: 'b'} FROM result)) THEN 'as expected' ELSE error('the two new points were not labelled a and b') END AS verdict;
  • The statement builds its own data.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the two new points were not labelled a and b”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

KNIME Analytics Platform (1)
  • K Nearest Neighborread 2026-09-27

    Classify test rows by majority vote of their k nearest training rows

RapidMiner Studio (1)
  • k-NNread 2026-09-27

    Train a k-nearest-neighbour model for classification or regression

Background

The k-nearest neighbours algorithm (k-NN) labels a new example with the label most frequent among the k training examples nearest to it.[1] In k-NN, the training phase consists only of storing the feature vectors and class labels of the training examples.[1] When k is 1, k-NN simply assigns the new example to the class of its single nearest neighbour.[1]

When to use it

Being a non-parametric method, nearest neighbours classification is often successful where the decision boundary between classes is very irregular.[2] The naive version of k-NN computes the distance from the new example to every stored example, which is computationally intensive for large training sets.[1] KNIME's K Nearest Neighbor node uses a K-dimensional tree (KD tree), yet is still only suited to a few thousand to ten thousand or so training instances.[3][2] Reach for k-NN when the labelled table is modest, its columns are few and numeric, and a reader should be able to see which rows decided each label. (Meridian's view)

Because k-NN relies on distance, feature-wise normalising of the training data can greatly improve its accuracy when features have different units or vastly different scales.[1] The accuracy of k-NN can be severely degraded by noisy or irrelevant features, or by feature scales that are not consistent with their importance.[1] KNIME's K Nearest Neighbor node uses all and only the numeric columns, with the Euclidean distance, and forwards the other columns of the rows it classifies as they are.[3] Rescale the columns to a common footing before measuring distance, and treat the choice of scaling and of columns as part of the model. (Meridian's view)

The best choice of k depends on the data: a larger k generally reduces the effect of noise but makes the boundaries between classes less distinct.[1][2] In two-class k-NN classification, choosing an odd k avoids tied votes.[1] In scikit-learn, if the k-th and (k+1)-th neighbours are at identical distances but have different labels, the result depends on the ordering of the training data.[2] In DuckDB, the mode aggregate returns the most frequent value and is affected by ordering.[4] Settle ties with a rule you name, such as the nearer neighbour or a fixed key, so the same data always gives the same label. (Meridian's view)

With basic majority voting, a skewed class distribution lets examples of the more frequent class dominate the prediction, because they tend to be common among the k nearest neighbours.[1] Weighting each neighbour's vote in proportion to the inverse of its distance from the new example is one way to overcome a skewed class distribution in k-NN.[1]

For high-dimensional data, with more than 10 dimensions for example, dimension reduction is usually performed before applying k-NN, to avoid the effects of the curse of dimensionality.[1] In the k-NN context, the curse of dimensionality means that Euclidean distance is unhelpful in high dimensions, because all vectors are almost equidistant from the query vector.[1]

The method

k-NN was first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover.[1] Thomas M. Cover and Peter E. Hart published Nearest neighbor pattern classification in IEEE Transactions on Information Theory in 1967.[1] As the amount of data approaches infinity, the two-class k-NN algorithm is guaranteed an error rate no worse than twice the Bayes error rate, the minimum achievable given the distribution of the data.[1]

In RapidMiner, the k-NN operator first finds the k training examples closest to the unknown example, then classifies it by a majority vote of the neighbours found.[5] A commonly used distance metric for continuous variables in k-NN is Euclidean distance, and for discrete variables, such as in text classification, an overlap metric such as Hamming distance can be used.[1] RapidMiner's k-NN operator offers a Mixed Euclidean Distance for data with both nominal and numerical attributes, taking a distance of 0 between equal nominal values and 1 otherwise.[5]

The plain k-NN classifier can be viewed as giving each of the k nearest neighbours a weight of 1/k and every other example a weight of 0, which weighted nearest neighbour classifiers generalise.[1] A common weighting scheme in k-NN gives each neighbour a weight of 1/d, where d is its distance from the new example.[1] scikit-learn's RadiusNeighborsClassifier instead votes among the neighbours within a fixed radius r, which can be a better choice where the data is not uniformly sampled.[2]

Compared with the alternatives

Neighbours-based classification does not attempt to construct a general internal model, but simply stores instances of the training data.[2] Choose k-NN knowing that the whole labelled table is the model, and every new row is a search of it. (Meridian's view)

k-NN can also be used for regression, where the prediction is the average of the values of the k nearest neighbours.[1] In RapidMiner, the k-Means operator performs clustering on an unlabelled data set and can be considered the equivalent of the k-NN algorithm for unsupervised learning.[5] Pick k-NN when you already hold labelled rows and want a label for new ones, and clustering when you hold no labels and want to find groups. (Meridian's view)

Sources

  1. [1]k-nearest neighbors algorithm - Wikipediasecondary source
  2. [2]1.6. Nearest Neighbors — scikit-learn 1.9.1 documentationdocumentation
  3. [3]K Nearest Neighbordocumentation
  4. [4]Aggregate Functions – DuckDBdocumentation
  5. [5]k-NN - Rapidminer Documentationdocumentation