# Split a table

Shape / table

split one table into several by a key or a condition

Page: <https://meridian.online/operations/split-a-table>

## Status

Runs as a SQL step.

arcform carries it as a `sql:` step today.

## The DuckDB step

one table split into a file per key

It reads the `orders` table the proofs share. Create it first:

```sql
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);
```

Then run the statement:

```sql
COPY (SELECT * FROM orders WHERE region IS NOT NULL) TO '<your_scratch_dir>/by_region' (FORMAT csv, PARTITION_BY (region));
CREATE TEMP TABLE result AS
SELECT count(DISTINCT regexp_extract(file, 'region=([^/]+)', 1)) AS branches, (SELECT count(*) FROM read_csv('<your_scratch_dir>/by_region/region=north/*.csv')) AS north_rows FROM glob('<your_scratch_dir>/by_region/*/*.csv');
SELECT * FROM result;
SELECT CASE WHEN ((SELECT branches = 4 AND north_rows = 3 FROM result)) THEN 'as expected' ELSE error('the table did not split into four branches with three rows in north') END AS verdict;
```

Replace `<your_scratch_dir>` with a directory of your own that you can write to before running the statement.

The statement writes `by_region` itself, in `<your_scratch_dir>`.

It needs nothing installed beyond DuckDB itself.

## Variants

- Split by the value of a key: Make one table per distinct value of a key, a hash of it, or a script's result.
- Split into chunks by size: Cut a table or file into pieces of a fixed row count or byte size, or at a delimiter.

## What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

- Apache NiFi: [`PartitionRecord`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.PartitionRecord/) (read 2026-09-27)
- Apache NiFi: [`ScriptedPartitionRecord`](https://nifi.apache.org/components/org.apache.nifi.processors.script.ScriptedPartitionRecord/) (read 2026-09-27)
- Apache NiFi: [`SegmentContent`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.SegmentContent/) (read 2026-09-27)
- Apache NiFi: [`SplitAvro`](https://nifi.apache.org/components/org.apache.nifi.processors.avro.SplitAvro/) (read 2026-09-27)
- Apache NiFi: [`SplitContent`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.SplitContent/) (read 2026-09-27)
- Apache NiFi: [`SplitExcel`](https://nifi.apache.org/components/org.apache.nifi.processors.excel.SplitExcel/) (read 2026-09-27)
- Apache NiFi: [`SplitPCAP`](https://nifi.apache.org/components/org.apache.nifi.processors.network.pcap.SplitPCAP/) (read 2026-09-27)
- Apache NiFi: [`SplitRecord`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.SplitRecord/) (read 2026-09-27)
- Apache NiFi: [`SplitText`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.SplitText/) (read 2026-09-27)
- Polars: [`DataFrame.partition_by`](https://docs.pola.rs/api/python/stable/reference/dataframe/modify_select.html) (read 2026-09-27)
- Power Query M: [`Table.Partition`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#row-operations) (read 2026-09-27)
- Power Query M: [`Table.Split`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#transformation) (read 2026-09-27)
- Power Query M: [`Table.SplitAt`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#row-operations) (read 2026-09-27)
- RapidMiner Studio: [`Group Into Collection`](https://docs.rapidminer.com/latest/studio/operators/extensions/Operator%20Toolbox/blending/table/group_into_collection.html) (read 2026-09-27)