Shape / inspect · All operations
Cross-tabulate two columns
A variant of Measure correlation
Count the rows for each combination of values in two or more categorical columns, with association tests such as chi-square.
Status
- In arcform
Runs as a SQL step
arcform carries it as a
sql:step today.
The DuckDB step
Cross-tabulate two categorical columns, with a chi-square statistic
It reads the orders table the proofs share. Create it first:
CREATE TABLE orders AS SELECT * FROM (VALUES
(1, 'Alice Ng', 'north', DATE '2026-01-03', 120.00, 'widget', 'a|b', '{"channel":"web","items":[1,2]}'),
(2, 'alice ng', 'north', DATE '2026-01-05', 80.50, 'gadget', 'b', '{"channel":"shop","items":[3]}'),
(3, 'Bob Okafor', 'south', DATE '2026-01-05', 300.00, 'widget', 'a|c', '{"channel":"web","items":[4,5,6]}'),
(4, 'Carla Díaz', 'east', DATE '2026-01-09', NULL, 'gizmo', NULL, '{"channel":"web","items":[]}'),
(5, 'Bob Okafor', 'south', DATE '2026-01-12', 45.25, 'gadget', 'c', '{"channel":"phone","items":[7]}'),
(6, 'Dev Patel', 'west', DATE '2026-01-12', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'),
(7, 'Carla Diaz', 'east', DATE '2026-01-20', 99.99, 'gizmo', 'b|c', '{"channel":"shop","items":[10]}'),
(8, 'Dev Patel', 'west', DATE '2026-02-02', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'),
(9, 'Eve Larsen', NULL, DATE '2026-02-03', 15.00, 'gadget', NULL, '{"channel":"web","items":[11]}'),
(10, 'Alice Ng', 'north', DATE '2026-02-10', 220.00, 'gizmo', 'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);Then run the statement:
-- a full grid of product by region, so a combination that never occurs shows as 0 and not as a missing row
CREATE TEMP TABLE crosstab AS
WITH o AS (SELECT product, coalesce(region, 'unknown') AS region FROM orders),
grid AS (SELECT p.product, r.region FROM (SELECT DISTINCT product FROM o) p CROSS JOIN (SELECT DISTINCT region FROM o) r)
SELECT g.product, g.region, count(o.product) AS n
FROM grid g LEFT JOIN o ON o.product = g.product AND o.region = g.region
GROUP BY g.product, g.region;
CREATE TEMP TABLE wide AS SELECT * FROM (PIVOT crosstab ON region USING sum(n) GROUP BY product) ORDER BY product;
-- Pearson chi-square: the sum over cells of (observed - expected)^2 / expected, with expected = row total * column total / N
CREATE TEMP TABLE chi AS
SELECT round(sum((n - expected) ^ 2 / expected), 4) AS chi_square,
(count(DISTINCT product) - 1) * (count(DISTINCT region) - 1) AS degrees_of_freedom,
round(sqrt(sum((n - expected) ^ 2 / expected) / (sum(n) * (least(count(DISTINCT product), count(DISTINCT region)) - 1))), 4) AS cramers_v
FROM (SELECT *, sum(n) OVER (PARTITION BY product) * sum(n) OVER (PARTITION BY region) / sum(n) OVER () AS expected FROM crosstab);
-- three ways: product by region by month, again on a full grid so empty combinations read 0
CREATE TEMP TABLE crosstab3 AS
WITH o AS (SELECT product, coalesce(region, 'unknown') AS region, strftime(order_date, '%Y-%m') AS month FROM orders),
grid AS (SELECT p.product, r.region, m.month
FROM (SELECT DISTINCT product FROM o) p CROSS JOIN (SELECT DISTINCT region FROM o) r CROSS JOIN (SELECT DISTINCT month FROM o) m)
SELECT g.product, g.region, g.month, count(o.product) AS n
FROM grid g LEFT JOIN o ON o.product = g.product AND o.region = g.region AND o.month = g.month
GROUP BY g.product, g.region, g.month;
SELECT * FROM wide;
SELECT * FROM chi;
SELECT * FROM crosstab3 WHERE n > 0 ORDER BY product, region, month;
SELECT CASE WHEN ((SELECT count(*) FROM crosstab) = 15 AND (SELECT sum(n) FROM crosstab) = 10 AND (SELECT count(*) FROM crosstab WHERE n = 0) = 7 AND (SELECT n FROM crosstab WHERE product = 'widget' AND region = 'west') = 2 AND (SELECT n FROM crosstab WHERE product = 'gizmo' AND region = 'east') = 2 AND (SELECT list(column_name) FROM (DESCRIBE wide)) = ['product', 'east', 'north', 'south', 'unknown', 'west'] AND (SELECT (chi_square, degrees_of_freedom, cramers_v) FROM chi) = (10.9722, 8, 0.7407) AND (SELECT count(*) FROM crosstab3) = 30 AND (SELECT sum(n) FROM crosstab3) = 10 AND (SELECT count(*) FROM crosstab3 WHERE n = 0) = 21 AND (SELECT n FROM crosstab3 WHERE product = 'gizmo' AND region = 'east' AND month = '2026-01') = 2 AND (SELECT n FROM crosstab3 WHERE product = 'widget' AND region = 'west' AND month = '2026-02') = 1 AND (SELECT count(*) FROM crosstab3 c JOIN (SELECT product, region, sum(n) AS n FROM crosstab3 GROUP BY product, region) m USING (product, region) JOIN crosstab x USING (product, region) WHERE m.n <> x.n) = 0) THEN 'as expected' ELSE error('the product by region counts were not 15 cells totalling 10 with 7 empty, or the chi-square was not 10.9722 on 8 degrees of freedom, or the product by region by month grid was not 30 cells with 21 empty') END AS verdict;- It needs nothing installed beyond DuckDB itself.
- The statement checks its own result and raises an error when it is wrong: “the product by region counts were not 15 cells totalling 10 with 7 empty, or the chi-square was not 10.9722 on 8 degrees of freedom, or the product by region by month grid was not 30 cells with 21 empty”.
- It passed on DuckDB
v2.0.0 (Cyanoptera) d54cbbe3df.
What other tools call it
Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).
Alteryx Designer (1)
Contingency Tableread 2026-09-27Cross-tabulate up to four fields to see how they relate
KNIME Analytics Platform (1)
Crosstabread 2026-09-27Cross-tabulate two columns into a contingency table with chi-square statistics
Background
A contingency table, or cross tabulation, is a table in a matrix format that displays the multivariate frequency distribution of the variables.[1] A contingency table gives a basic picture of the interrelation between two variables and can help find interactions between them.[1] Contingency tables are heavily used in survey research, business intelligence, engineering and scientific research.[1]
When to use it
A test of independence assesses whether observations on two variables, expressed in a contingency table, are independent of each other.[2] The tests that compare a contingency table's proportions, such as Pearson's chi-squared test and Fisher's exact test, apply provided the entries represent individuals randomly sampled from the population the conclusions are about.[1] A table of every order a business took is a census of its own records rather than a sample; there the counts describe the data, and a p-value adds little. (Meridian's view)
Pearson's chi-squared test assumes a sufficiently large sample, and on a smaller one it yields an inaccurate inference.[2] A common rule for Pearson's chi-squared test asks for expected counts of 5 or more in all cells of a 2-by-2 table and in 80% of cells in larger tables, with no cell expected to be zero.[2] Merge thin categories, or turn to an exact test, before reading a chi-squared statistic from a table full of small counts. (Meridian's view)
In pandas, crosstab with the default dropna=True drops categories that have no observed instances in the input.[3] In DuckDB, a PIVOT on several columns makes a column for every combination of their distinct values, and a combination not present in the data is populated with NULL.[4] Build the full grid of categories before counting, so that a combination that never happened reads as 0 rather than vanishing or showing as missing. (Meridian's view) A contingency table may have more than two variables, but higher order contingency tables are difficult to represent visually.[1]
The method
In DuckDB, the PIVOT statement separates the distinct values of a column into columns of their own, and without a USING clause it aggregates with count(*).[4] A contingency table's row and column totals are called marginal totals, and the total of all observations is the grand total.[1]
Under the hypothesis of independence, the expected frequency of a cell is the total sample size times its row's fraction of the observations times its column's fraction.[2] Pearson's chi-squared statistic sums, over every cell, the squared difference between the observed and expected frequencies divided by the expected frequency.[2] For a test of independence, the degrees of freedom are (rows − 1) × (columns − 1), counting the categories of each variable.[2] In SciPy, chi2_contingency applies Yates' correction for continuity by default when the degrees of freedom is 1, adjusting each observed value by 0.5 towards its expected value.[5] The properties of Pearson's chi-squared test were first investigated by Karl Pearson in 1900.[2] Karl Pearson first used the term contingency table in On the Theory of Contingency and Its Relation to Association and Normal Correlation, published in 1904.[1]
Cramér's V, published by Harald Cramér in 1946, is a measure of association between two nominal variables based on Pearson's chi-squared statistic, giving a value between 0 and 1.[6] Cramér's V is the square root of the chi-squared statistic divided by the sample size and by the smaller of the table's two dimensions minus one.[6] Cramér's V can reach 1 only when each variable is completely determined by the other.[6] Cramér's V is symmetrical, so it does not matter which variable is placed in the rows and which in the columns.[6]
Compared with the alternatives
A pivot table is a way to create contingency tables using spreadsheet software.[1] The simplest measure of association for a 2 × 2 contingency table is the odds ratio.[1]
Sources
- [1]Contingency table - Wikipediasecondary source
- [2]Pearson's chi-squared test - Wikipediasecondary source
- [3]pandas.crosstab — pandas 3.1.1 documentationdocumentation
- [4]PIVOT Statement – DuckDBdocumentation
- [5]chi2_contingency — SciPy v1.18.0 Manualdocumentation
- [6]Cramér's V - Wikipediasecondary source