Find the topics in a body of text

Status

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Background

When to use it

Topic models are commonly used to organise and discover latent features in large collections of unstructured text.^[1] LDA can discover topics in a collection of documents and then classify each document by how relevant it is to each discovered topic.^[2] LDA finds topics from the co-occurrence of terms, but assigning a meaningful label to a topic is up to the user and often requires specialised knowledge.^[2] Expect to name the topics yourself: read each topic's top words and a few of its documents before giving it a label. (Meridian's view)

LDA and probabilistic latent semantic analysis (pLSA) both require the user to specify the number of topics before training starts, as with k-means clustering.^[2] Try several numbers of topics and keep the one whose topics you can name, rather than taking the first number tried. (Meridian's view)

In RapidMiner's LDA operators, results may differ between runs unless the reproducible setting, which turns off parallel execution, is on.^[3][4] Fix the random seed and record it with the topics, so a later run can be compared with this one. (Meridian's view) Alteryx suggests considering its Text Pre-processing tool upstream of its Topic Modeling tool.^[5] Alteryx's Topic Modeling tool ignores a word that appears in more than a maximum share of documents, with 80% as the recommended setting.^[5]

The method

LDA is a generative statistical model that explains how a collection of text documents can be described by a set of unobserved topics.^[2] LDA assumes that documents are a random mixture of latent topics and that each topic is characterised by a probability distribution over words.^[2] LDA assumes that most documents contain only a relatively small number of topics.^[2] Learning LDA's latent topics from a corpus is typically done by Bayesian inference, often with Gibbs sampling or variational Bayes.^[2]

An early topic model was described by Papadimitriou, Raghavan, Tamaki and Vempala in 1998.^[1] pLSA was created by Thomas Hofmann in 1999, and LDA is a generalisation of pLSA.^[1] LDA was first presented as a graphical model for population genetics by J. K. Pritchard, M. Stephens and P. Donnelly in 2000.^[2] LDA was applied in machine learning by David Blei, Andrew Ng and Michael I. Jordan in 2003.^[2]

RapidMiner's LDA operators use the ParallelTopicModel of the Mallet library with the SparseLDA sampling scheme.^[3][4] RapidMiner's LDA operators add a topic to each document with the confidence that the document belongs to it, and return the top 5 most used words of each topic.^[3][4]

BERTopic's default clustering step can identify outliers, so documents are not forced into a cluster where they might not belong.^[6] In BERTopic, the topic numbered -1 holds all outliers and should typically be ignored.^[7]

Sources

1. [1]Topic model - Wikipediasecondary source
2. [2]Latent Dirichlet allocation - Wikipediasecondary source
3. [3]Extract Topics from Documents (LDA) - Rapidminer Documentationdocumentation
4. [4]Extract Topics from Data (LDA) - Rapidminer Documentationdocumentation
5. [5]Topic Modelingdocumentation
6. [6]The Algorithm - BERTopicdocumentation
7. [7]BERTopicdocumentation