# Pack columns into a nested value

Shape / columns

A variant of [Parse nested values](https://meridian.online/operations/parse-nested-values).

Combine several columns of each row into one struct, list, JSON, XML or Avro column.

Page: <https://meridian.online/operations/parse-nested-values/pack-columns-into-nested-value>

## Status

Runs as a SQL step.

arcform carries it as a `sql:` step today, with the `avro` extension.

## The DuckDB step

pack several columns of each row into one struct, list, JSON, XML and Avro column

```sql
CREATE TEMP TABLE items AS SELECT * FROM (VALUES
  (1, 'Alice Ng', 120.00, 'north', 'widget'), (2, 'Tom & <Jerry>', NULL, NULL, 'gizmo')
) t(id, customer, amount, region, product);
-- each text is escaped (& first, then < and >) before it is put between tags; a missing value gives an empty tag
CREATE MACRO esc(x) AS replace(replace(replace(x, '&', '&amp;'), '<', '&lt;'), '>', '&gt;');
CREATE TEMP TABLE result AS
SELECT id,
       {'customer': customer, 'amount': amount} AS as_struct,
       [region, product] AS as_list,
       to_json({'customer': customer, 'amount': amount}) AS as_json,
       '<row><customer>' || coalesce(esc(customer), '') || '</customer><amount>' || coalesce(amount::VARCHAR, '') || '</amount></row>' AS as_xml
FROM items;
SELECT * FROM result ORDER BY id;
DESCRIBE result;
SELECT CASE WHEN ((SELECT list(column_name) FROM (DESCRIBE result)) = ['id', 'as_struct', 'as_list', 'as_json', 'as_xml'] AND (SELECT as_struct.customer || '/' || as_struct.amount FROM result WHERE id = 1) = 'Alice Ng/120.00' AND (SELECT as_list FROM result WHERE id = 2) = [NULL, 'gizmo'] AND (SELECT as_json::VARCHAR FROM result WHERE id = 2) = '{"customer":"Tom & <Jerry>","amount":null}' AND (SELECT as_xml FROM result WHERE id = 2) = '<row><customer>Tom &amp; &lt;Jerry&gt;</customer><amount></amount></row>' AND (SELECT as_xml FROM result WHERE id = 1) = '<row><customer>Alice Ng</customer><amount>120.00</amount></row>') THEN 'as expected' ELSE error('customer and amount were not packed into one struct, one JSON and one escaped XML column, or region and product into one list') END AS verdict;

-- Avro: the packed struct is written to an Avro file and read back as one column
COPY (SELECT id, as_struct FROM result) TO '<your_scratch_dir>/packed.avro' (FORMAT avro);
SELECT CASE WHEN ((SELECT count(*) FROM read_avro('<your_scratch_dir>/packed.avro') WHERE as_struct.customer = 'Tom & <Jerry>' AND as_struct.amount IS NULL) = 1 AND (SELECT count(*) FROM read_avro('<your_scratch_dir>/packed.avro')) = 2) THEN 'as expected' ELSE error('the packed struct did not come back from the Avro file as one column') END AS verdict;
```

Replace `<your_scratch_dir>` with a directory of your own that you can write to before running the statement.

The statement writes `packed.avro` itself, in `<your_scratch_dir>`.

It needs the `avro` extension.

## What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

- Alteryx Designer: [`JSON Build`](https://help.alteryx.com/current/en/designer/tools/laboratory/json-build-tool.html) (read 2026-09-27)
- Apache DataFusion: [`Struct Functions`](https://datafusion.apache.org/user-guide/sql/scalar_functions.html) (read 2026-09-27)
- Duckle: [`Merge Objects`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/website/docs/components.html) (read 2026-09-25)
- EasyMorph: [`Construct JSON`](https://help.easymorph.com/doku.php?id=transformations:constructjson) (read 2026-09-27)
- Apache Hop: [`Add XML`](https://hop.apache.org/manual/latest/pipeline/transforms/addxml.html) (read 2026-09-27)
- Apache Hop: [`Avro encode`](https://hop.apache.org/manual/latest/pipeline/transforms/avro-encode.html) (read 2026-09-27)
- Ibis: [`Array`](https://github.com/ibis-project/ibis/blob/05d2b293344e86cb41c1a426071734f7e270335a/ibis/expr/operations/arrays.py) (read 2026-09-27)
- Ibis: [`StructColumn`](https://github.com/ibis-project/ibis/blob/05d2b293344e86cb41c1a426071734f7e270335a/ibis/expr/operations/structs.py) (read 2026-09-27)
- KNIME Analytics Platform: [`Column to XML`](https://api.hub.knime.com/nodes/*VqkSwDBKksqolRKh?details=full) (read 2026-09-27)
- KNIME Analytics Platform: [`Columns to Collection`](https://api.hub.knime.com/nodes/*eNgVOpmMFNJq3Jie?details=full) (read 2026-09-27)
- KNIME Analytics Platform: [`Columns to JSON`](https://api.hub.knime.com/nodes/*S5sWr2sm-SjOA14S?details=full) (read 2026-09-27)
- KNIME Analytics Platform: [`JSON Column Combiner`](https://api.hub.knime.com/nodes/*YWufNYG_cCOXV2VP?details=full) (read 2026-09-27)
- KNIME Analytics Platform: [`Nominal Probability Distribution Creator`](https://api.hub.knime.com/nodes/*TD93rewy2eMlDiSJ?details=full) (read 2026-09-27)
- KNIME Analytics Platform: [`XML Column Combiner`](https://api.hub.knime.com/nodes/*iY4Ln0a0nzW73ykY?details=full) (read 2026-09-27)
- Apache NiFi: [`AttributesToCSV`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.AttributesToCSV/) (read 2026-09-27)
- Apache NiFi: [`AttributesToJSON`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.AttributesToJSON/) (read 2026-09-27)
- Pentaho Data Integration: [`Add XML`](https://docs.pentaho.com/pdia-data-integration/9.3-data-integration/pdi-transformation-steps-reference-overview) (read 2026-09-27)
- Polars: [`DataFrame.to_struct`](https://docs.pola.rs/api/python/stable/reference/dataframe/export.html) (read 2026-09-27)
- Polars: [`concat_arr`](https://docs.pola.rs/api/python/stable/reference/expressions/functions.html) (read 2026-09-27)
- Polars: [`concat_list`](https://docs.pola.rs/api/python/stable/reference/expressions/functions.html) (read 2026-09-27)
- Polars: [`list`](https://docs.pola.rs/api/python/stable/reference/expressions/functions.html) (read 2026-09-27)
- Polars: [`struct`](https://docs.pola.rs/api/python/stable/reference/expressions/functions.html) (read 2026-09-27)
- Power Query M: [`Table.CombineColumnsToRecord`](https://learn.microsoft.com/en-us/powerquery-m/table-functions#transformation) (read 2026-09-27)
- RapidMiner Studio: [`Data To JSON`](https://docs.rapidminer.com/latest/studio/operators/extensions/Text%20Processing/utility/data_to_json.html) (read 2026-09-27)
- Talend: [`tWriteAvroFields`](https://help.qlik.com/talend/en-US/components/8.0/processing/processing-component) (read 2026-09-27)
- Talend: [`tWriteDynamicFields`](https://help.qlik.com/talend/en-US/components/8.0/processing/processing-component) (read 2026-09-27)
- Talend: [`tWriteJSONField`](https://help.qlik.com/talend/en-US/components/8.0/processing/processing-component) (read 2026-09-27)
- Talend: [`tWriteXMLField`](https://help.qlik.com/talend/en-US/components/8.0/xml/xml-component) (read 2026-09-27)
- Talend: [`tWriteXMLFields`](https://help.qlik.com/talend/en-US/components/8.0/processing/processing-component) (read 2026-09-27)
- Trifacta: [`Nest Transform`](https://help.alteryx.com/aac/en/trifacta-classic/wrangle-language/language-appendices/transforms.html) (read 2026-09-27)
- Trifacta: [`Nest columns`](https://help.alteryx.com/aac/en/trifacta-classic/reference/transformation-reference.html) (read 2026-09-27)
- VisiData: [`contract-source-cols`](https://github.com/saulpw/visidata/blob/7609212ec2febafa9caba7183cbebfaf49627909/visidata/features/expand_cols.py#L196) (read 2026-09-23)
