Show how one input moves the prediction

Status

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Background

When to use it

Partial dependence is a global method: it considers all instances and makes a statement about the global relationship of a feature with the predicted outcome.^[1] In practice a partial dependence plot covers one feature or at most two, because one feature produces a 2D plot and two produce a 3D plot.^[1] Rank the inputs by importance first, then draw the partial dependence of the few that matter most. (Meridian's view)

Partial dependence assumes that the features of interest are independent of the other features, and this assumption is often violated in practice.^[2][1] For the partial dependence of a person's height at 200 cm, averaging over the marginal distribution of weight might include a weight below 50 kg, which is unrealistic for a 2 metre person.^[1] Partial dependence plots show only the average marginal effect, so if half the data has a positive association with the prediction and half a negative one, the curve could be a horizontal line.^[1] Read a flat partial dependence line as no average effect, not as no effect. (Meridian's view) A partial dependence plot that omits the feature's distribution can mislead, because regions with almost no data may be overinterpreted, and a rug or histogram on the axis solves it.^[1] The partial dependence calculation intervenes on a feature and measures the change in predictions, so the relationship it shows is causal for the model but not necessarily for the real world.^[1]

The method

In scikit-learn, the partial dependence at a point is the expected response over the complement features, the input features other than those of interest.^[2] The scikit-learn 'recursion' method, supported only by some tree-based estimators, follows the branch for a feature of interest at each split and otherwise follows both branches, weighted by the fraction of training samples in each.^[2]

For a categorical feature, partial dependence gives one estimate per category, found by forcing every data instance to that category and averaging the predictions.^[1] For a classifier that outputs probabilities, partial dependence shows the probability for a certain class, and multiple classes can be drawn as one line or plot per class.^[1]

Compared with the alternatives

An individual conditional expectation (ICE) plot visualises the dependence of the prediction on a feature for each sample separately, with one line per sample, where a partial dependence plot shows the average effect.^[2] In scikit-learn, the average of the ICE lines corresponds to the partial dependence line.^[2] Goldstein, Kapelner, Bleich and Pitkin presented ICE plots as a refinement of partial dependence plots, because with substantial interaction effects an average curve can obscure a heterogeneous relationship.^[3] Accumulated local effect (ALE) plots, which work with the conditional instead of the marginal distribution, are one solution to the problem of correlated features in partial dependence plots.^[1] SHapley Additive exPlanations (SHAP) dependence plots are an alternative to partial dependence and ALE plots, showing the variance of effects as well as their average.^[4] Greenwell, Boehmke and McCarthy proposed a partial dependence-based importance measure in 2018, in which a flat curve marks an unimportant feature and the more the curve varies, the more important the feature.^[1] The partial dependence-based importance captures only the main effect of a feature and ignores interactions, so a feature that acts mainly through interactions can have a flat curve.^[1]

Sources

1. [1]19 Partial Dependence Plot (PDP) – Interpretable Machine Learningsecondary source
2. [2]5.1. Partial Dependence and Individual Conditional Expectation plots — scikit-learn 1.9.1 documentationdocumentation
3. [3][1309.6392] Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectationsecondary source
4. [4]18 SHAP – Interpretable Machine Learningsecondary source