# Profile a table Shape / inspect summary statistics, distinct counts, null counts Page: ## Status Runs as a SQL step. arcform carries it as a `sql:` step today. ## The DuckDB step per-column summary statistics It reads the `orders` table the proofs share. Create it first: ```sql CREATE TABLE orders AS SELECT * FROM (VALUES (1, 'Alice Ng', 'north', DATE '2026-01-03', 120.00, 'widget', 'a|b', '{"channel":"web","items":[1,2]}'), (2, 'alice ng', 'north', DATE '2026-01-05', 80.50, 'gadget', 'b', '{"channel":"shop","items":[3]}'), (3, 'Bob Okafor', 'south', DATE '2026-01-05', 300.00, 'widget', 'a|c', '{"channel":"web","items":[4,5,6]}'), (4, 'Carla Díaz', 'east', DATE '2026-01-09', NULL, 'gizmo', NULL, '{"channel":"web","items":[]}'), (5, 'Bob Okafor', 'south', DATE '2026-01-12', 45.25, 'gadget', 'c', '{"channel":"phone","items":[7]}'), (6, 'Dev Patel', 'west', DATE '2026-01-12', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'), (7, 'Carla Diaz', 'east', DATE '2026-01-20', 99.99, 'gizmo', 'b|c', '{"channel":"shop","items":[10]}'), (8, 'Dev Patel', 'west', DATE '2026-02-02', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'), (9, 'Eve Larsen', NULL, DATE '2026-02-03', 15.00, 'gadget', NULL, '{"channel":"web","items":[11]}'), (10, 'Alice Ng', 'north', DATE '2026-02-10', 220.00, 'gizmo', 'a|b|c', '{"channel":"phone","items":[12,13]}') ) t(order_id, customer, region, order_date, amount, product, tags, meta); ``` Then run the statement: ```sql CREATE TEMP TABLE result AS SELECT column_name, approx_unique, null_percentage, "min", "max", count FROM (SUMMARIZE orders); SELECT * FROM result; SELECT CASE WHEN ((SELECT count(*) = 8 AND max(null_percentage) FILTER (WHERE column_name = 'amount') = 10 AND max("max") FILTER (WHERE column_name = 'order_id') = '10' AND max(count) = 10 FROM result)) THEN 'as expected' ELSE error('the profile does not report 10 rows with a tenth of amounts missing') END AS verdict; ``` It needs nothing installed beyond DuckDB itself. ## Variants - Classify columns by what they hold: Label each column with a semantic or personal-data type from the share of values matching each pattern. - Count distinct values: Count the distinct values of each column, or the distinct rows, exactly or approximately. - Count missing values in every column: Count the blank or null cells, or the filled ones, in each column. - Profile other kinds of content: Profile text, images or behaviour segments rather than table columns. - Score each column for quality: Score columns for missing share, stability, entropy, ID-ness or overall health. - Count the rows or test for an empty table: Return the number of rows, or whether there are none, as one value. - Summarise every column: Give one row per column with its type, counts, nulls, distinct values, minimum, maximum, mean and quartiles. ## What other tools call it Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2). - Alteryx Designer: [`Basic Data Profile`](https://help.alteryx.com/current/en/designer/tools/data-investigation/basic-data-profile-tool.html) (read 2026-09-27) - Alteryx Designer: [`Compare Behavior`](https://help.alteryx.com/current/en/designer/tools/behavior-analysis/compare-behavior-tool.html) (read 2026-09-27) - Alteryx Designer: [`Data Health`](https://help.alteryx.com/current/en/designer/tools/alteryx-intelligence-suite/machine-learning/data-health.html) (read 2026-09-27) - Alteryx Designer: [`Detail Fields`](https://help.alteryx.com/current/en/designer/tools/behavior-analysis/detail-fields-tool.html) (read 2026-09-27) - Alteryx Designer: [`Field Summary`](https://help.alteryx.com/current/en/designer/tools/data-investigation/field-summary-tool.html) (read 2026-09-27) - Alteryx Designer: [`Image Profile`](https://help.alteryx.com/current/en/designer/tools/alteryx-intelligence-suite/computer-vision/image-profile.html) (read 2026-09-27) - Apache DataFusion: [`describe`](https://docs.rs/datafusion/latest/datafusion/dataframe/struct.DataFrame.html) (read 2026-09-27) - DuckDB: [`SUMMARIZE`](https://duckdb.org/docs/current/sql/statements/summarize) (read 2026-09-27) - DuckDB: `summary` (read 2026-09-27) - Duckle: [`Classify / PII Detect`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/frontend/src/workflow-ui/palette-data.ts) (read 2026-09-25) - Duckle: [`Column Profile`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/website/docs/components.html) (read 2026-09-25) - Duckle: [`Column Profile (Advanced)`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/frontend/src/workflow-ui/palette-data.ts) (read 2026-09-25) - AWS Glue: [`AllStatistics`](https://docs.aws.amazon.com/glue/latest/dg/dqdl-rule-types.html) (read 2026-09-27) - AWS Glue: [`Entropy`](https://docs.aws.amazon.com/glue/latest/dg/dqdl-rule-types.html) (read 2026-09-27) - AWS Glue: [`count`](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame.html) (read 2026-09-27) - KNIME Analytics Platform: [`Statistics`](https://api.hub.knime.com/nodes/*7N4LxTHoPJrHN5lT?details=full) (read 2026-09-27) - KNIME Analytics Platform: [`Statistics View`](https://api.hub.knime.com/nodes/*-VN57hNSccwmJlDo?details=full) (read 2026-09-27) - mlpack: [`mlpack_preprocess_describe`](https://www.mlpack.org/doc/user/bindings/cli.html) (read 2026-09-27) - Apache NiFi: [`CountText`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.CountText/) (read 2026-09-27) - OpenRefine: [`Blank records per column`](https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/views/data-table/data-table-view.js) (read 2026-09-27) - OpenRefine: [`Blank values per column`](https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/views/data-table/data-table-view.js) (read 2026-09-27) - OpenRefine: [`Non-blank records per column`](https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/views/data-table/data-table-view.js) (read 2026-09-27) - OpenRefine: [`Non-blank values per column`](https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/views/data-table/data-table-view.js) (read 2026-09-27) - Pentaho Data Integration: [`Univariate Statistics`](https://docs.pentaho.com/pdia-data-integration/9.3-data-integration/pdi-transformation-steps-reference-overview) (read 2026-09-27) - Polars: [`DataFrame.approx_n_unique`](https://docs.pola.rs/api/python/stable/reference/dataframe/descriptive.html) (read 2026-09-27) - Polars: [`DataFrame.describe`](https://docs.pola.rs/api/python/stable/reference/dataframe/descriptive.html) (read 2026-09-27) - Polars: [`DataFrame.is_empty`](https://docs.pola.rs/api/python/stable/reference/dataframe/descriptive.html) (read 2026-09-27) - Polars: [`DataFrame.n_unique`](https://docs.pola.rs/api/python/stable/reference/dataframe/descriptive.html) (read 2026-09-27) - Polars: [`DataFrame.null_count`](https://docs.pola.rs/api/python/stable/reference/dataframe/descriptive.html) (read 2026-09-27) - Power Query M: [`Table.ApproximateRowCount`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#information) (read 2026-09-27) - Power Query M: [`Table.IsEmpty`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#information) (read 2026-09-27) - Power Query M: [`Table.Profile`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#information) (read 2026-09-27) - Power Query M: [`Table.RowCount`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#information) (read 2026-09-27) - RapidMiner Studio: [`Extract Statistics`](https://docs.rapidminer.com/latest/studio/operators/extensions/Operator%20Toolbox/blending/extract_statistics.html) (read 2026-09-27) - RapidMiner Studio: [`Quality Measures`](https://docs.rapidminer.com/latest/studio/operators/cleansing/quality_measures.html) (read 2026-09-27) - RapidMiner Studio: [`Statistics`](https://docs.rapidminer.com/latest/studio/operators/cleansing/data_statistics.html) (read 2026-09-27) - VisiData: [`describe-all`](https://github.com/saulpw/visidata/blob/7609212ec2febafa9caba7183cbebfaf49627909/visidata/features/describe.py#L116) (read 2026-09-23) - VisiData: [`describe-selected`](https://github.com/saulpw/visidata/blob/7609212ec2febafa9caba7183cbebfaf49627909/visidata/features/describe.py#L117) (read 2026-09-23) - VisiData: [`describe-sheet`](https://github.com/saulpw/visidata/blob/7609212ec2febafa9caba7183cbebfaf49627909/visidata/features/describe.py#L115) (read 2026-09-23) - Yggdrasil Decision Forests: [`infer_dataspec`](https://github.com/google/yggdrasil-decision-forests/blob/0a54b57c95ff0cf42ecf0310ace0513a4f32cb65/yggdrasil_decision_forests/cli/infer_dataspec.cc) (read 2026-09-27) - Yggdrasil Decision Forests: [`show_dataspec`](https://github.com/google/yggdrasil-decision-forests/blob/0a54b57c95ff0cf42ecf0310ace0513a4f32cb65/yggdrasil_decision_forests/cli/show_dataspec.cc) (read 2026-09-27)