Model / explain · All operations

Inspect a model's structure

A variant of Explain a model

Read out a model's internals, such as its coefficients, trees, rules, centroids or formula.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (2)
  • Model Coefficientsread 2026-09-27

    Extract coefficient names and values from a regression model into a table

  • Variance Inflation Factorsread 2026-09-27

    Report variance inflation factors for each variable of a regression model

KNIME Analytics Platform (5)
RapidMiner Studio (4)
  • Cluster Model Visualizerread 2026-09-27

    Visual summary of a centroid cluster model: heat map, centroid charts, cluster descriptions

  • Create Formularead 2026-09-27

    Extract the prediction formula from a model that can express one

  • Extract Cluster Prototypesread 2026-09-27

    Output the centroids or prototypes of a cluster model as a table

  • Visualize Model by SOMread 2026-09-27

    Plot a model's predictions over a 2-D self-organising map of the data

Yggdrasil Decision Forests (9)
  • GBT model accessors (group)read 2026-09-27

    3 accessors: num_trees_per_iteration, early_stopping_triggered, activation

  • describeread 2026-09-27

    Render a text or HTML description of the model, its features, trees and training logs

  • get_tree / get_all_trees / iter_treesread 2026-09-27

    Read trees out of the model as editable Python objects

  • in_bag_example_indicesread 2026-09-27

    Return the row indices bootstrap-sampled to train one tree, replayed from the training seed

  • num_examples_per_treeread 2026-09-27

    Return how many rows were sampled to grow each isolation tree

  • num_trees / num_nodesread 2026-09-27

    Count the trees and nodes in the model

  • print_treeread 2026-09-27

    Print one tree as indented text

  • show_modelread 2026-09-27

    Print a model's features, label, variable importances, structure, engines and training logs

  • winner_takes_allread 2026-09-27

    Report whether trees vote one class each or average their class distributions

Background

Model-specific interpretation analyses parts of a model to understand it better, such as the Gini importance in random forests.[1] Reading out a model's coefficients or weights means looking inside it, which model-agnostic methods, working only with inputs and outputs, do not do.[1] A small decision tree can be visualised and understood easily, and so can a linear regression model with not too many coefficients.[1]

When to use it

Models that are interpretable by design, meaning inherently interpretable models such as linear regression, are usually easier to debug and improve, because they give insights into their inner workings.[1] Interpretable models often faithfully explain how predictions were made, and tend to make it easier to check with domain experts that the model is consistent with domain knowledge.[1] A regression model with hundreds of features may not be entirely interpretable, yet its individual coefficients can still be interpreted, as can the individual rules of a huge decision list.[1] Interpreting a model in place of the data requires assuming that the model structure reflects the world.[1]

Decision trees are very interpretable as long as they are short, since the maximum number of terminal nodes in a binary tree is 2 to the power of the depth.[2] Decision trees are also quite unstable: a few changes in the training data can create a completely different tree.[2] When two features of a linear regression are strongly correlated, it becomes indeterminable to which of them the effects should be attributed.[3] The variance inflation factor (VIF) measures how much the variance of an estimated regression coefficient is increased because of collinearity.[4] Put the features on a common scale before comparing coefficients, and look at how stable they are across resamples before trusting one. (Meridian's view)

The method

A linear regression model predicts the target as a weighted sum of the feature inputs, and its first weight, the intercept, is not multiplied with a feature.[3] In a linear regression, an increase of a numerical feature by one unit increases the prediction by the feature's weight when all other feature values remain fixed.[3] For a binary feature in a linear regression, one value counts as the reference category, and changing to the other value changes the estimated outcome by the feature's weight.[3] Estimated weights come with confidence intervals, valid when the linear regression model is the correct model for the data.[3]

The classification and regression trees (CART) algorithm is probably the most popular algorithm for growing a decision tree.[2] In CART, the prediction in each leaf node is the average outcome of the training data in that node.[2] A decision tree is read from the root node, where the edges say which subsets of the data are being looked at, down to a leaf node, which gives the predicted outcome, with all the edges connected by AND.[2]

Compared with the alternatives

Interpretability by design means training inherently interpretable models, such as a logistic regression instead of a random forest, while post-hoc interpretability uses a method after the model is trained.[1] If a linear regression is interpreted through its coefficients, switching to a rule-based classifier also changes the means of interpretation, while a model-agnostic method stays the same when the model changes.[1] When a model with better predictive performance exists, interpreting a weaker interpretable model requires arguing why it represents the data correctly.[1]

Sources

  1. [1]4 Methods Overview – Interpretable Machine Learningsecondary source
  2. [2]9 Decision Tree – Interpretable Machine Learningsecondary source
  3. [3]6 Linear Regression – Interpretable Machine Learningsecondary source
  4. [4]Variance inflation factor - Wikipediasecondary source