Model / evaluate · All operations

Draw ROC and lift curves

A variant of Evaluate a model

Rank rows by score and report gain, lift or ROC points at each cut-off.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Rank rows by score and report gain, lift and ROC points at each cut-off

SQL
-- ten scored rows: a score and whether the row was truly positive; 0.80 is a tie between a negative and a positive
CREATE TEMP TABLE scored AS SELECT * FROM (VALUES
  (1, 0.95, 1), (2, 0.90, 1), (3, 0.80, 0), (4, 0.80, 1), (5, 0.70, 1), (6, 0.60, 0), (7, 0.50, 1), (8, 0.40, 0), (9, 0.30, 0), (10, 0.20, 0)) t(id, score, actual);
-- one row per distinct score: flag every row scoring at least the cut-off, so tied rows enter together
CREATE TEMP TABLE cutoffs AS
SELECT score AS cutoff,
       sum(sum(actual)) OVER w AS tp, sum(sum(1 - actual)) OVER w AS fp, sum(count(*)) OVER w AS flagged
FROM scored GROUP BY score WINDOW w AS (ORDER BY score DESC ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW);
CREATE TEMP TABLE curve AS
SELECT cutoff, tp::INTEGER AS tp, fp::INTEGER AS fp,
       tp / (SELECT sum(actual) FROM scored) AS gain,                      -- share of all positives found (the true positive rate)
       fp / (SELECT sum(1 - actual) FROM scored) AS fpr,
       flagged / (SELECT count(*) FROM scored) AS share_flagged,
       (tp / flagged) / ((SELECT sum(actual) FROM scored) / (SELECT count(*) FROM scored)) AS lift   -- precision over the base rate
FROM cutoffs;
SELECT cutoff, tp, fp, round(gain, 3) AS gain, round(fpr, 3) AS fpr, round(share_flagged, 2) AS share_flagged, round(lift, 3) AS lift FROM curve ORDER BY cutoff DESC;
-- area under the ROC curve by the trapezoid rule, starting from the origin (0, 0), and by counting positive-negative pairs the positive wins (a tie counts half)
CREATE TEMP TABLE area AS
WITH steps AS (SELECT fpr - coalesce(lag(fpr) OVER o, 0) AS width, gain + coalesce(lag(gain) OVER o, 0) AS height FROM curve WINDOW o AS (ORDER BY cutoff DESC))
SELECT (SELECT sum(width * height / 2) FROM steps) AS trapezoid,
       (SELECT sum(CASE WHEN p.score > n.score THEN 1.0 WHEN p.score = n.score THEN 0.5 ELSE 0 END) / (count(DISTINCT p.id) * count(DISTINCT n.id))
        FROM scored p, scored n WHERE p.actual = 1 AND n.actual = 0) AS pairs;
SELECT * FROM area;
SELECT CASE WHEN ((SELECT count(*) = 9 AND list([tp, fp] ORDER BY cutoff DESC) = [[1, 0], [2, 0], [3, 1], [4, 1], [4, 2], [5, 2], [5, 3], [5, 4], [5, 5]]
    AND list(round(gain, 3) ORDER BY cutoff DESC) = [0.2, 0.4, 0.6, 0.8, 0.8, 1.0, 1.0, 1.0, 1.0]
    AND list(round(fpr, 3) ORDER BY cutoff DESC) = [0.0, 0.0, 0.2, 0.2, 0.4, 0.4, 0.6, 0.8, 1.0]
    AND list(round(lift, 3) ORDER BY cutoff DESC) = [2.0, 2.0, 1.5, 1.6, 1.333, 1.429, 1.25, 1.111, 1.0] FROM curve)
  AND (SELECT abs(trapezoid - 0.86) < 1e-12 AND abs(pairs - 0.86) < 1e-12 FROM area)) THEN 'as expected' ELSE error('the curve should have 9 cut-offs from 0.95 to 0.20 with gain reaching 1 at 0.50, lift 2 at the first two, and an area of 0.86 by the trapezoid and by pairs') END AS verdict;
  • The statement builds its own data.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the curve should have 9 cut-offs from 0.95 to 0.20 with gain reaching 1 at 0.50, lift 2 at the first two, and an area of 0.86 by the trapezoid and by pairs”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Lift Chartread 2026-09-27

    Draw cumulative gains and incremental response charts to compare binary classifiers

KNIME Analytics Platform (6)
RapidMiner Studio (3)
  • Compare ROCsread 2026-09-27

    Plot ROC curves of several inner learners on one chart for comparison

  • Create Lift Chartread 2026-09-27

    Lift chart from discretised confidences with a Pareto overlay

  • Lift Chart (Simple)read 2026-09-27

    Lift chart of model gain over random guessing for each confidence segment

Background

A receiver operating characteristic (ROC) curve is a plot that shows how a binary classifier performs at varying threshold values.[1] A ROC curve plots the true positive rate (TPR) against the false positive rate (FPR) at each threshold setting.[1] Lift is the response rate a targeting model achieves within its target, divided by the average response rate of the population as a whole.[2]

When to use it

ROC analysis provides tools to select possibly optimal models and discard suboptimal ones independently from, and before specifying, the cost context or the class distribution.[1] A random guess gives a point on the diagonal from the bottom-left to the top-right corner of the ROC plot, the line of no-discrimination, regardless of the base rates of positives and negatives.[1] The true positive rate is also known as sensitivity, and the false positive rate equals one minus specificity.[1] The area under the ROC curve (AUC) summarises sensitivity and specificity, but does not inform regarding precision and negative predictive value.[1] A ROC AUC of 0.9 might correspond to a precision of 0.2 and a negative predictive value of 0.1.[1] Where only part of the curve matters, such as the low false positive rates often of prime interest in population screening, a partial AUC can be computed.[1]

In RapidMiner, a lift chart is particularly useful where a cost-benefit ratio can be optimised, as often happens in marketing, and can show that reaching 80% of respondents takes only 30% of the address base.[3]

The method

The ROC curve was first developed by electrical and radar engineers during World War II for detecting enemy objects in battlefields, starting in 1941, which led to its name.[1] In RapidMiner's Compare ROCs, the curve is calculated by ordering the classified examples by confidence and taking them in decreasing confidence, plotting the false positive rate against the true positive rate.[4] For examples that share a confidence, RapidMiner's Compare ROCs can take the correct classifications first (optimistic), the wrong ones first (pessimistic) or alternate them (neutral).[4] RapidMiner's optimistic, neutral and pessimistic ROC curves are the same if no examples share a confidence or all examples with equal confidence are assigned to the same class.[4] Settle how tied scores are handled before comparing two models' curves. (Meridian's view)

The AUC equals the probability that a classifier ranks a randomly chosen positive instance higher than a randomly chosen negative one.[1] The AUC is closely related to the Mann–Whitney U, which tests whether positives are ranked higher than negatives.[1] Twice the area between the ROC curve and the no-discrimination line is called the Gini coefficient, especially in credit scoring.[1]

Alteryx's Lift Chart tool groups rows into 10 deciles ordered by predicted probability, and its cumulative response chart shows the share of total response captured by contacting the best 10 percent, 20 percent and so on.[5] In RapidMiner's Lift Chart (Simple), a random model covering 30% of the population would find only 30% of the target class.[3]

Compared with the alternatives

Like the precision-recall curve, the lift curve shows a tradeoff similar to the ROC curve's, and it is also similar to the Lorenz or power curve of econometrics.[2] Whereas an uninformative classifier has a ROC AUC of 0.5, alternative measures such as informedness put chance performance at 0 and perfect performance at 1.[1]

Sources

  1. [1]Receiver operating characteristic - Wikipediasecondary source
  2. [2]Lift (data mining) - Wikipediasecondary source
  3. [3]Lift Chart (Simple) - Rapidminer Documentationdocumentation
  4. [4]Compare ROCs - Rapidminer Documentationdocumentation
  5. [5]Lift Chart Tooldocumentation