Model / tune · All operations

Tune a model

Hyperparameter search, feature selection, feature weighting.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

Shape of the result
a model or a file, not a table
Level
needs an arcform operator

Variants

Weight columns by importance

Give each column a weight for its relevance to the label or its share of variance.

Drop weak columns by a measure

Drop columns that are highly correlated, near constant or below a weight threshold, without training a model.

Tune a decision threshold

Find the confidence threshold that minimises cost or meets a required recall.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

Alteryx Designer (2)
  • Importance Weightsread 2026-09-27

    Rank candidate predictors by strength of relationship to a target, for variable selection

  • Stepwiseread 2026-09-27

    Select the best predictor subset for a regression model by stepwise search

KNIME Analytics Platform (7)
mlpack (1)
  • Hyperparameter tuningread 2026-09-27

    HyperParameterTuner: search hyperparameters of any mlpack method, by grid or by gradient

RapidMiner Studio (45)
Yggdrasil Decision Forests (9)
  • BackwardSelectionFeatureSelectorread 2026-09-27

    Drop the least important features round by round and keep the best-scoring set

  • FeatureSelectorLogsread 2026-09-27

    Table of selection rounds with the features kept and the score reached

  • HYPERPARAMETER_OPTIMIZERread 2026-09-27

    Meta-learner that trains a base learner repeatedly over a search space and keeps the best model

  • OptimizerLogsread 2026-09-27

    Table of tuning trials with their hyperparameters and scores

  • RandomSearchTunerread 2026-09-27

    Sample hyperparameter sets at random from a search space and keep the best trial

  • VizierTunerread 2026-09-27

    Tune hyperparameters through Google Vizier; documented as for Googlers only

  • choiceread 2026-09-27

    Add one hyperparameter and its candidate values to a tuner's search space

  • feature_selection_logsread 2026-09-27

    Return each feature-selection round's features and score, if selection was run

  • hyperparameter_optimizer_logsread 2026-09-27

    Return each tuning trial's hyperparameters and score, if the model was tuned

Background

Model selection is the task of choosing a model from among various candidates on the basis of a performance criterion, to pick the best one.[1] In machine learning, algorithmic approaches to model selection include feature selection and hyperparameter optimisation.[1] Hyperparameters are settings of a model's learning process, named in contrast to parameters, which are characteristics the model learns from the data.[2]

When to use it

Optimal hyperparameter values are not always easy to predict, and a separate process of hyperparameter tuning is often needed to find a suitable combination for the data and task.[2] Most performance variation can be attributed to just a few hyperparameters.[2] In scikit-learn, it is common for a small subset of an estimator's parameters to have a large impact on its performance while the others can be left at their default values.[3] Spend the search on the few settings that move the score, and leave the rest at their defaults until there is a reason not to. (Meridian's view)

Cross-validation is often used to estimate how well a set of hyperparameters will generalise, and so to choose the values that maximise that estimate.[4] Selecting hyperparameters by their score on a validation set risks overfitting them to that set, so the same score cannot also estimate the final model's performance.[4] The final model's generalisation performance has to be measured on data independent of the tuning, either a second test set or an outer cross-validation called nested cross-validation.[4] In scikit-learn's example on the iris data, choosing the parameters that maximise non-nested cross-validation biases the model to the dataset and yields an overly optimistic score.[5] Report a tuned model's score from data the tuning never touched, and treat the search's own best score as a ranking, not a forecast. (Meridian's view)

In scikit-learn, feature selection, like any other preprocessing, should use only the training data, because including the test data optimistically biases the model.[6] In scikit-learn's demonstration with randomly generated features and random targets, feature selection on all the data gives an accuracy score much higher than chance.[6] For binary classification, scikit-learn predicts the positive class when the conditional probability is greater than 0.5, a hard-coded rule that is not ideal for most use cases.[7] Where a missed positive costs more than a false alarm, choose the decision threshold from those costs rather than keeping the default. (Meridian's view)

The method

A feature selection algorithm can be seen as a search technique that proposes feature subsets, combined with an evaluation measure that scores them.[8] The evaluation measure is what separates the three main categories of feature selection algorithms: wrappers, filters and embedded methods.[8] Filter methods score a feature subset with a proxy measure chosen to be fast to compute, such as mutual information or the Pearson product-moment correlation coefficient, instead of a model's error rate.[8] Many filters give a ranking of features rather than an explicit best subset, and the cut-off point in the ranking is chosen via cross-validation.[8] Redundancy and irrelevance are distinct notions in feature selection, since one relevant feature may be redundant in the presence of another relevant feature with which it is strongly correlated.[8] In scikit-learn, VarianceThreshold removes every feature whose variance does not meet a threshold, and by default removes features that have the same value in all samples.[9]

Compared with the alternatives

Feature extraction creates new features from functions of the original features, whereas feature selection finds a subset of the features.[8]

Sources

  1. [1]Model selection - Wikipediasecondary source
  2. [2]Hyperparameter (machine learning) - Wikipediasecondary source
  3. [3]3.2. Tuning the hyper-parameters of an estimator — scikit-learn 1.9.1 documentationdocumentation
  4. [4]Hyperparameter optimization - Wikipediasecondary source
  5. [5]Nested versus non-nested cross-validation — scikit-learn 1.9.1 documentationdocumentation
  6. [6]12. Common pitfalls and recommended practices — scikit-learn 1.9.1 documentationdocumentation
  7. [7]3.3. Tuning the decision threshold for class prediction — scikit-learn 1.9.1 documentationdocumentation
  8. [8]Feature selection - Wikipediasecondary source
  9. [9]1.13. Feature selection — scikit-learn 1.9.1 documentationdocumentation