Model / supervised · All operations

Train a random forest

A variant of Train a tree model

Train many trees on random samples of rows and columns and average their votes.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Forest Modelread 2026-09-27

    Train a random forest of decision trees on random samples of the data

KNIME Analytics Platform (4)
mlpack (2)
  • RandomForestread 2026-09-27

    Parallelised random forest classifier

  • mlpack_random_forest_trainread 2026-09-27

    Train a random forest classifier from two tables, with nclasses, ntrees, seed and threads; store JSON model

RapidMiner Studio (1)
  • Random Forestread 2026-09-27

    Train a random forest for classification or regression

Talend (2)
  • tMatchModelread 2026-09-27

    Train a Random Forest matching model from labelled suspect pairs, for tMatchPredict

  • tRandomForestModelread 2026-09-27

    Train a random forest classification model from feature vectors

Yggdrasil Decision Forests (1)
  • RANDOM_FORESTread 2026-09-27

    Train a bagged forest of deep trees with out-of-bag evaluation; Python class RandomForestLearner

Background

Random forests are an ensemble learning method for classification, regression and other tasks that works by creating many decision trees during training.[1] For classification, a random forest outputs the class selected by most trees; for regression, it outputs the average of the trees' predictions.[1] Random forests correct for decision trees' habit of overfitting to their training set.[1]

When to use it

Trees grown very deep tend to overfit their training sets, with low bias but very high variance, and random forests average many deep trees trained on different parts of the same training set to reduce the variance.[1][2] Averaging the trees comes at the expense of a small increase in bias and some loss of interpretability, but generally greatly boosts the performance of the final model.[1] If features are linearly correlated with the target, a random forest may not enhance the accuracy of its base learner, the decision tree.[1] The basic random forest procedure may not work well when there are a large number of features but only a small proportion of them are informative.[1] One remedy for a table of many uninformative features is prefiltering, which eliminates features that are mostly just noise.[1]

Typically, a few hundred to several thousand trees are used in a random forest, depending on the size and nature of the training set.[1] In scikit-learn, more trees in a forest are better but take longer to compute, and results stop getting significantly better beyond a critical number of trees.[2] In scikit-learn, fully grown forest trees often give good results, but might result in models that consume a lot of RAM.[2] Fix the random seed when the same forest must come out of two runs. (Meridian's view)

The default impurity-based feature importance of a scikit-learn random forest favours high-cardinality features, and uses training statistics, so it does not reflect a feature's usefulness on a test set.[1] Read a forest's impurity-based importances as a hint about which columns the trees leaned on, not as proof of which columns matter. (Meridian's view)

The method

The first algorithm for random decision forests was created in 1995 by Tin Kam Ho using the random subspace method.[1] Bootstrap aggregating, or bagging, repeatedly selects a random sample with replacement of the training set, a bootstrap sample, and fits a tree to each sample.[1][2] An extension of Ho's algorithm, developed by Leo Breiman and Adele Cutler, combines Breiman's bagging idea with random selection of features.[1] Out-of-bag error is the mean prediction error on each training sample, using only the trees that did not have that sample in their bootstrap sample.[1]

Random forests also select a random subset of the features at each candidate split, a process sometimes called feature bagging.[1][2] Feature bagging is used because one or a few very strong predictors would otherwise be selected in many of the trees, making the trees correlated.[1] KNIME's Random Forest Learner builds each tree from a bootstrapped set of rows the same size as the input table, and considers sqrt(m) randomly selected attributes at each split, where m is the number of learning columns.[3] In contrast to Breiman's original publication, scikit-learn's random forest combines trees by averaging their probabilistic predictions, instead of letting each tree vote for a single class.[2] In scikit-learn, bootstrap sampling lets a forest's generalisation error be estimated on the out-of-bag samples, by setting oob_score=True.[2]

Compared with the alternatives

Extremely randomized trees, or ExtraTrees, differ from a random forest in training each tree on the whole learning sample, and in choosing random cut-points for each candidate feature instead of the locally optimal one.[1] In scikit-learn, extremely randomized trees usually reduce the variance of the model a bit more than a random forest, at the expense of a slightly greater increase in bias.[2] In scikit-learn, random forests typically rely on deep trees that overfit individually, while gradient boosting models build shallow trees that underfit individually and are faster to fit and predict.[2] Try a forest and a boosted model on the same held-out rows and compare them, rather than choosing between them on reputation. (Meridian's view)

Sources

  1. [1]Random forest - Wikipediasecondary source
  2. [2]1.11. Ensembles: Gradient boosting, random forests, bagging, voting, stacking — scikit-learn 1.9.1 documentationdocumentation
  3. [3]Random Forest Learnerdocumentation