Shape / columns · All operations

Encode categories by the target

A variant of Encode categories as numbers

Replace each category with a statistic of the label for that category, such as its mean or weight of evidence.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Replace each category with the mean of the label and its weight of evidence

SQL
CREATE TEMP TABLE labelled AS SELECT * FROM (VALUES
  (1, 'A', 1), (2, 'A', 1), (3, 'A', 0), (4, 'A', 1),
  (5, 'B', 0), (6, 'B', 0), (7, 'B', 1),
  (8, 'C', 0), (9, 'C', 1),
  (10, 'D', 1), (11, 'E', 0)
) t(id, category, label);
-- weight of evidence is undefined for a category with no events or no non-events (D and E), so it is NULL there and the
-- mean label, which is defined, still encodes them
CREATE TEMP TABLE result AS
WITH per_category AS (
  SELECT category, avg(label) AS mean_label, sum(label) AS events, count(*) - sum(label) AS non_events FROM labelled GROUP BY category),
totals AS (SELECT sum(events) AS events, sum(non_events) AS non_events FROM per_category)
SELECT l.id, l.category, l.label, round(c.mean_label, 4) AS target_mean,
       round(CASE WHEN c.events = 0 OR c.non_events = 0 THEN NULL
                  ELSE ln((c.events / t.events) / (c.non_events / t.non_events)) END, 4) AS woe
FROM labelled l JOIN per_category c USING (category) CROSS JOIN totals t;
SELECT * FROM result ORDER BY id;
SELECT CASE WHEN ((SELECT count(*) FROM result) = 11 AND (SELECT [target_mean, woe] FROM result WHERE id = 1) = [0.75, 0.9163] AND (SELECT [target_mean, woe] FROM result WHERE id = 5) = [0.3333, -0.8755] AND (SELECT [target_mean, woe] FROM result WHERE id = 9) = [0.5, -0.1823] AND (SELECT count(DISTINCT woe) FROM result WHERE category = 'A') = 1 AND (SELECT [target_mean::VARCHAR, woe::VARCHAR] FROM result WHERE id = 10) = ['1.0', NULL] AND (SELECT [target_mean::VARCHAR, woe::VARCHAR] FROM result WHERE id = 11) = ['0.0', NULL]) THEN 'as expected' ELSE error('the categories were not replaced by their mean label and weight of evidence, or a category with one label did not get a mean and no weight of evidence') END AS verdict;
  • The statement builds its own data.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the categories were not replaced by their mean label and weight of evidence, or a category with one label did not get a mean and no weight of evidence”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

RapidMiner Studio (2)
  • Target Encodingread 2026-09-27

    Replace nominal categories with the label's probability or mean per category, with smoothing

  • Weight of Evidenceread 2026-09-27

    Bin numeric columns and replace each bin with its weight-of-evidence value against the label

Background

Target encoding replaces each category with a statistic of the label for it: the probability of the label when it is categorical, its average when numeric.[1] Target encoding replaces a categorical column with one numeric column, or with one column per class when the label has more than two classes.[1]

When to use it

Target encoding suits categorical features with high cardinality, such as zip code or region, where one-hot encoding would inflate the feature space.[2] Target encoding carries a risk of overfitting, and running it before the validation split leaks the label into the encoding.[1] Learn the encoding on the training rows alone and apply it to every other row, as you would any model. (Meridian's view)

scikit-learn's TargetEncoder uses cross fitting in fit_transform: the training data is split into k folds, and each fold is encoded with encodings learnt on the other k-1 folds.[2] Cross fitting keeps target information from leaking into the training representation, especially for high-cardinality features whose categories each appear only a few times.[2] In scikit-learn's TargetEncoder, fit(X, y).transform(X) does not equal fit_transform(X, y).[2][3] scikit-learn's TargetEncoder treats missing values as a category of their own, and encodes categories not seen during fit with the overall target mean.[2]

The method

scikit-learn's TargetEncoder encodes each category with a shrunk estimate that mixes the target mean within the category with the global target mean.[3] scikit-learn's TargetEncoder weights the category's mean by a shrinkage factor ni / (m + ni), where ni is the category's row count and m is a smoothing factor.[2] A larger smoothing factor in scikit-learn's TargetEncoder puts more weight on the global mean.[2][3] With smooth set to auto, scikit-learn's TargetEncoder sets the smoothing factor to an empirical Bayes estimate: the variance of y within the category over the global variance of y.[2] scikit-learn's TargetEncoder is documented with Daniele Micci-Barreca's July 2001 paper, A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems, as a reference.[2]

RapidMiner's Weight of Evidence operator bins a numeric attribute and gives each bin the value ln(% of negatives / % of positives), from the bin's share of the data set's negatives and positives.[4] Say which outcome sits in the numerator of a weight of evidence, so that a reader can tell what a positive code means. (Meridian's view)

Compared with the alternatives

One-hot encoding is an unsupervised encoding better suited to low-cardinality categorical variables, because it makes one new feature per unique category.[3] With more than two classes, target encoding makes one column per class, and since the number of classes is typically small, far fewer columns than one-hot encoding.[1] In a benchmark by Florian Pargent and colleagues, regularised versions of target encoding consistently gave the best results across five machine learning algorithms.[5] Integer encoding was less effective than regularised target encoding in the benchmark of high-cardinality encodings by Florian Pargent and colleagues.[5]

Sources

  1. [1]Target Encoding - Rapidminer Documentationdocumentation
  2. [2]8.3. Preprocessing data — scikit-learn 1.9.1 documentationdocumentation
  3. [3]TargetEncoder — scikit-learn 1.9.1 documentationdocumentation
  4. [4]Weight of Evidence - Rapidminer Documentationdocumentation
  5. [5][2104.00629] Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality featuressecondary source