Model / evaluate · All operations

Evaluate a clustering

A variant of Evaluate a model

Score a clustering by density, distance, entropy or agreement with known labels.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Score a clustering by density, distance, entropy and agreement with known labels

SQL
-- ten points with the cluster a model gave each and the label they really carry
CREATE TEMP TABLE pts AS SELECT * FROM (VALUES
  (1, 1.0, 1.0, 0, 'a'), (2, 1.2, 0.8, 0, 'a'), (3, 0.9, 1.1, 0, 'a'), (4, 1.1, 1.2, 0, 'a'), (5, 3.0, 3.0, 0, 'b'),
  (6, 5.0, 5.0, 1, 'b'), (7, 5.2, 4.8, 1, 'b'), (8, 4.9, 5.1, 1, 'b'), (9, 5.1, 5.2, 1, 'b'), (10, 3.2, 3.0, 1, 'a')) t(id, x, y, cluster, label);
CREATE TEMP TABLE centroid AS SELECT cluster, avg(x) AS cx, avg(y) AS cy FROM pts GROUP BY cluster;
CREATE TEMP TABLE radial AS SELECT p.cluster, p.id, sqrt(power(p.x - c.cx, 2) + power(p.y - c.cy, 2)) AS r FROM pts p JOIN centroid c USING (cluster);
-- distance: within-cluster sum of squares, and the silhouette of every point (mean distance to the other clusters against its own)
CREATE TEMP TABLE silhouette AS
WITH d AS (SELECT a.id, a.cluster = b.cluster AS same, sqrt(power(a.x - b.x, 2) + power(a.y - b.y, 2)) AS dist FROM pts a JOIN pts b ON a.id <> b.id),
     m AS (SELECT id, avg(dist) FILTER (WHERE same) AS a, avg(dist) FILTER (WHERE NOT same) AS b FROM d GROUP BY id)
SELECT avg((b - a) / greatest(a, b)) AS mean_silhouette FROM m;
-- density: points per unit of area of the circle round the centroid that reaches the farthest member
CREATE TEMP TABLE density AS SELECT cluster, count(*) / (pi() * power(max(r), 2)) AS points_per_area FROM radial GROUP BY cluster;
-- entropy of the true labels inside each cluster (bits), weighted by cluster size; 0 would be a pure cluster
CREATE TEMP TABLE entropy AS
WITH share AS (SELECT cluster, label, count(*)::DOUBLE / sum(count(*)) OVER (PARTITION BY cluster) AS p, sum(count(*)) OVER (PARTITION BY cluster) AS n FROM pts GROUP BY cluster, label)
SELECT sum(n * -p * log2(p)) / (SELECT count(*) FROM pts) AS weighted_entropy FROM share;
-- agreement with the labels: the Rand index and the adjusted Rand index from the cluster-by-label table
CREATE TEMP TABLE agreement AS
WITH cell AS (SELECT cluster, label, count(*) AS n FROM pts GROUP BY cluster, label),
     tot AS (SELECT sum(n * (n - 1) / 2) AS together, (SELECT sum(n * (n - 1) / 2) FROM (SELECT cluster, sum(n) AS n FROM cell GROUP BY cluster)) AS in_cluster,
                    (SELECT sum(n * (n - 1) / 2) FROM (SELECT label, sum(n) AS n FROM cell GROUP BY label)) AS in_label, (SELECT count(*) * (count(*) - 1) / 2 FROM pts) AS pairs FROM cell)
SELECT (pairs + 2 * together - in_cluster - in_label) / pairs AS rand_index,
       (together - in_cluster * in_label / pairs) / ((in_cluster + in_label) / 2 - in_cluster * in_label / pairs) AS adjusted_rand FROM tot;
CREATE TEMP TABLE result AS
SELECT (SELECT sum(r * r) FROM radial) AS within_ss, (SELECT mean_silhouette FROM silhouette) AS silhouette,
       (SELECT list(round(points_per_area, 9) ORDER BY cluster) FROM density) AS densities,
       (SELECT weighted_entropy FROM entropy) AS entropy, rand_index, adjusted_rand FROM agreement;
SELECT * FROM result;
-- reference values were computed outside DuckDB: the two cluster means are (1.44, 1.42) and (4.68, 4.62); each cluster has entropy 0.7219 bits
-- (four labels of one kind and one of the other); tolerance 1e-9 is for float rounding only
SELECT CASE WHEN ((SELECT abs(within_ss - 12.456) < 1e-9 AND abs(silhouette - 0.6295099535) < 1e-9
    AND densities = [0.322829499, 0.330553591] AND abs(entropy - 0.7219280949) < 1e-9
    AND abs(rand_index - 0.6444444444) < 1e-9 AND abs(adjusted_rand - 0.28) < 1e-9 FROM result)) THEN 'as expected' ELSE error('the clustering should score 12.456 within-cluster squares, a silhouette of 0.6295, entropy 0.7219 bits, a Rand index of 0.6444 and an adjusted Rand index of 0.28') END AS verdict;
  • The statement builds its own data.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the clustering should score 12.456 within-cluster squares, a silhouette of 0.6295, entropy 0.7219 bits, a Rand index of 0.6444 and an adjusted Rand index of 0.28”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
KNIME Analytics Platform (1)
  • Entropy Scorerread 2026-09-27

    Score a clustering against a reference clustering by entropy and quality measures

RapidMiner Studio (5)

Background

A clustering can be evaluated internally by a quality score, externally against a ground truth classification, manually by an expert, or by its usefulness.[1] Internal evaluation measures usually give the best score to clusters with high similarity within a cluster and low similarity between clusters.[1] External evaluation measures how close a clustering is to benchmark classes, sets of pre-classified items often created by expert humans.[1]

When to use it

Evaluation of clustering results is as difficult as the clustering itself.[1] External evaluation needs ground truth labels, yet with such labels there would be no need to cluster, and in practical applications they are usually missing.[1] Ground truth labels reflect only one possible partitioning of the data set, which does not imply that no different, and perhaps better, clustering exists.[1]

Many internal measures, among them the Davies–Bouldin index and the silhouette coefficient, rest on the intuition that items in one cluster should be more similar than items in different clusters.[1] Internal evaluation is biased towards algorithms that use the same cluster model: k-means clustering optimises object distances, so a distance-based internal criterion will likely overrate its result.[1] Internal measures are best suited to showing where one algorithm performs better than another, which does not imply that one algorithm produces more valid results.[1] Score a clustering with a measure built on a different idea of a cluster from the one the algorithm optimised, and look at the clusters themselves before trusting any figure. (Meridian's view)

The method

The silhouette was proposed by the Belgian statistician Peter Rousseeuw in 1987.[2] The silhouette value measures how similar an object is to its own cluster compared with other clusters, and ranges from −1 to +1, a high value meaning a good match to its own cluster.[2] An average silhouette width over 0.7 is considered strong, over 0.5 reasonable and over 0.25 weak, though high values become harder to reach as the dimensionality of the data grows.[2] The silhouette score is specialised for convex-shaped clusters and may not perform well if the clusters have irregular shapes or varying sizes.[2]

The Davies–Bouldin index was introduced by David L. Davies and Donald W. Bouldin in 1979.[3] Clusters with low distances within them and high distances between them have a low Davies–Bouldin index, so the clustering with the smallest index is considered best by that criterion.[1] RapidMiner's Cluster Distance Performance evaluates centroid-based clusterings, such as those of k-means and k-medoids, by average within-cluster distance and the Davies–Bouldin index.[4]

The Rand index, named after William M. Rand, compares all pairs of elements and counts how often two clusterings agree, either both placing a pair in the same cluster or both placing it in different clusters.[5] The Rand index has a value between 0 and 1, where 1 means the two clusterings are exactly the same.[5] The adjusted Rand index (ARI) is the Rand index corrected for the similarity expected by chance, and is commonly used to compare a clustering with known class labels.[5] Unlike the Rand index, the ARI can be negative, when the index is less than the expected index.[5] In scikit-learn, the unadjusted Rand index does not ensure a value close to 0.0 for a random labelling, while the adjusted Rand index corrects for chance and gives that baseline.[6] Scikit-learn's clustering metrics ignore the absolute values of cluster labels: permuting 0 and 1 or renaming 2 to 3 in the predicted labels gives the same score.[6]

Sources

  1. [1]Cluster analysis - Wikipediasecondary source
  2. [2]Silhouette (clustering) - Wikipediasecondary source
  3. [3]Davies–Bouldin index - Wikipediasecondary source
  4. [4]Cluster Distance Performance - Rapidminer Documentationdocumentation
  5. [5]Rand index - Wikipediasecondary source
  6. [6]2.3. Clustering — scikit-learn 1.9.1 documentationdocumentation