Give each row a membership per cluster

Status

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Background

When to use it

Soft clustering gives each object a degree of belonging to each cluster, such as a likelihood of belonging to it, where hard clustering only puts an object in a cluster or not.^[2] Reach for memberships when the rows that sit between groups matter, such as a customer close to two segments, rather than forcing each row into one. (Meridian's view)

In bioinformatics, clustering groups genes with similar expression patterns, and fuzzy clustering lets one gene belong to more than one such cluster.^[1] In gene expression analysis, fuzzy clustering is held to be more appropriate than hard clustering, because one gene may encode a protein that has more than one function.^[1] Any table whose rows can have a foot in several groups at once, such as products sold to several markets or documents on several topics, is a candidate for memberships. (Meridian's view)

The method

One of the most widely used fuzzy clustering algorithms is fuzzy c-means (FCM).^[1] FCM was developed by J.C. Dunn in 1973 and improved by J.C. Bezdek in 1981.^[1] FCM is very similar to the k-means algorithm, which assigns each observation to the cluster with the nearest mean, except that in k-means the membership values are either zero or one.^[1][3] FCM starts by choosing a number of clusters and randomly assigning each data point coefficients giving its degree of being in each cluster.^[1] FCM then repeatedly computes each cluster's centre, or centroid, and each point's coefficients, until the coefficients change by no more than a given sensitivity threshold between two iterations.^[1] In FCM, the centroid of a cluster is the mean of all points, weighted by their degree of belonging to the cluster.^[1] In FCM, a parameter m greater than 1 sets the degree of fuzziness, a larger m giving fuzzier clusters, and without experimentation or domain knowledge m is commonly set to 2.^[1] FCM aims to minimise an objective function that sums, over every point and cluster, the point's membership raised to the power m times its squared distance from the cluster centre.^[1] As m approaches 1, FCM's memberships converge to 0 or 1, and the FCM objective function coincides with that of k-means.^[1] FCM has the same problems as k-means: the minimum it finds is a local minimum, and its results depend on the initial choice of weights.^[1]

In KNIME Analytics Platform, the Fuzzy c-Means node returns the input table extended with each row's membership in each cluster, and a second table holding the cluster centres.^[4] In KNIME's Fuzzy c-Means node, the Fuzzifier option indicates how much the clusters are allowed to overlap.^[4] KNIME's Fuzzy c-Means node asks for normalised input data to obtain better clustering results.^[4] In KNIME's Fuzzy c-Means node, using the same random seed gives identical clustering results across executions.^[4] Record the fuzzifier and the seed alongside the membership columns, so that the same memberships can be produced again. (Meridian's view)

A Gaussian mixture model (GMM) is a probabilistic model that assumes all the data points are generated from a mixture of a finite number of Gaussian distributions with unknown parameters.^[5] In scikit-learn, fitting a GMM by expectation-maximisation (EM) computes for each point a probability of being generated by each Gaussian component of the model, then adjusts the parameters to maximise the likelihood of the data.^[5] In scikit-learn, repeating the EM steps for a GMM is guaranteed to converge, but only to a local optimum.^[5] GMMs trained with the EM algorithm maintain probabilistic assignments to clusters, instead of deterministic assignments.^[3] In scikit-learn, the GaussianMixture class always uses all the components it has access to, so how many components to use has to be decided by other means.^[5] In scikit-learn, the Bayesian Information Criterion (BIC) can be used to select the number of components in a GMM.^[5] In mlpack, a C++ machine learning library, mlpack_gmm_train trains a GMM with the EM algorithm, and mlpack_gmm_probability computes the probability that given points came from a given GMM.^[6] Before reading a probability column as a membership, check whether it holds one number per cluster or a single number for the whole model. (Meridian's view)

Compared with the alternatives

FCM is a soft version of k-means, in which each data point has a fuzzy degree of belonging to each cluster.^[3] Compared with FCM, using a mixture of Gaussians with the EM algorithm is a more statistically formalised method that includes the idea of partial membership in classes.^[1] To obtain a hard clustering from a Gaussian mixture, objects are often assigned to the Gaussian distribution they most likely belong to, a step a soft clustering does not need.^[2]

Sources

1. [1]Fuzzy clustering - Wikipediasecondary source
2. [2]Cluster analysis - Wikipediasecondary source
3. [3]k-means clustering - Wikipediasecondary source
4. [4]Fuzzy c-Meansdocumentation
5. [5]2.1. Gaussian mixture models — scikit-learn 1.9.1 documentationdocumentation
6. [6]mlpack documentationdocumentation