Shape / columns · All operations

Pack columns into a nested value

A variant of Parse nested values

Combine several columns of each row into one struct, list, JSON, XML or Avro column.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today, with the avro extension.

The DuckDB step

Pack several columns of each row into one struct, list, JSON, XML and Avro column

SQL
CREATE TEMP TABLE items AS SELECT * FROM (VALUES
  (1, 'Alice Ng', 120.00, 'north', 'widget'), (2, 'Tom & <Jerry>', NULL, NULL, 'gizmo')
) t(id, customer, amount, region, product);
-- each text is escaped (& first, then < and >) before it is put between tags; a missing value gives an empty tag
CREATE MACRO esc(x) AS replace(replace(replace(x, '&', '&amp;'), '<', '&lt;'), '>', '&gt;');
CREATE TEMP TABLE result AS
SELECT id,
       {'customer': customer, 'amount': amount} AS as_struct,
       [region, product] AS as_list,
       to_json({'customer': customer, 'amount': amount}) AS as_json,
       '<row><customer>' || coalesce(esc(customer), '') || '</customer><amount>' || coalesce(amount::VARCHAR, '') || '</amount></row>' AS as_xml
FROM items;
SELECT * FROM result ORDER BY id;
DESCRIBE result;
SELECT CASE WHEN ((SELECT list(column_name) FROM (DESCRIBE result)) = ['id', 'as_struct', 'as_list', 'as_json', 'as_xml'] AND (SELECT as_struct.customer || '/' || as_struct.amount FROM result WHERE id = 1) = 'Alice Ng/120.00' AND (SELECT as_list FROM result WHERE id = 2) = [NULL, 'gizmo'] AND (SELECT as_json::VARCHAR FROM result WHERE id = 2) = '{"customer":"Tom & <Jerry>","amount":null}' AND (SELECT as_xml FROM result WHERE id = 2) = '<row><customer>Tom &amp; &lt;Jerry&gt;</customer><amount></amount></row>' AND (SELECT as_xml FROM result WHERE id = 1) = '<row><customer>Alice Ng</customer><amount>120.00</amount></row>') THEN 'as expected' ELSE error('customer and amount were not packed into one struct, one JSON and one escaped XML column, or region and product into one list') END AS verdict;

-- Avro: the packed struct is written to an Avro file and read back as one column
COPY (SELECT id, as_struct FROM result) TO '<your_scratch_dir>/packed.avro' (FORMAT avro);
SELECT CASE WHEN ((SELECT count(*) FROM read_avro('<your_scratch_dir>/packed.avro') WHERE as_struct.customer = 'Tom & <Jerry>' AND as_struct.amount IS NULL) = 1 AND (SELECT count(*) FROM read_avro('<your_scratch_dir>/packed.avro')) = 2) THEN 'as expected' ELSE error('the packed struct did not come back from the Avro file as one column') END AS verdict;

Replace <your_scratch_dir> with a directory of your own that you can write to before running the statement.

  • The statement writes packed.avro itself, in <your_scratch_dir>.
  • It needs the avro extension.
  • The statement checks its own result and raises an error when it is wrong: “customer and amount were not packed into one struct, one JSON and one escaped XML column, or region and product into one list; then the packed struct did not come back from the Avro file as one column”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • JSON Buildread 2026-09-27

    Rebuild JSON text from the name and value rows of JSON Parse

Apache DataFusion (1)
  • Struct Functionsread 2026-09-27

    3 functions, 1 of them aliases: named_struct, row, struct

Apache Hop (2)
  • Add XMLread 2026-09-27

    Encode several fields of a row as an XML fragment in a new string column

  • Avro encoderead 2026-09-27

    Encode chosen fields into a single Avro record column

Apache NiFi (2)
  • AttributesToCSVread 2026-09-27

    Serialise FlowFile attributes as a CSV line, written to an attribute or to content

  • AttributesToJSONread 2026-09-27

    Serialise FlowFile attributes as a JSON object, written to an attribute or to content

Duckle (1)
EasyMorph (1)
Ibis (2)
  • Arrayread 2026-09-27

    Build an array value from a list of expressions

  • StructColumnread 2026-09-27

    Build a struct column from named expressions

KNIME Analytics Platform (6)
Pentaho Data Integration (1)
  • Add XMLread 2026-09-27

    Encode several fields into an XML fragment

Polars (5)
  • DataFrame.to_structread 2026-09-27

    Pack the columns of each row into one struct value, giving a Series

  • concat_arrread 2026-09-27

    Join several columns row by row into one fixed-size array column

  • concat_listread 2026-09-27

    Join several columns row by row into one list column

  • listread 2026-09-27

    Collect several columns into a list column, one element per expression

  • structread 2026-09-27

    Pack several columns into one struct column

Power Query M (1)
RapidMiner Studio (1)
  • Data To JSONread 2026-09-27

    Build nested JSON documents from the rows of a table

Talend (5)
  • tWriteAvroFieldsread 2026-09-27

    Serialise rows into Avro binaries for components that need serialised input

  • tWriteDynamicFieldsread 2026-09-27

    Combine input columns into one output column of Dynamic type

  • tWriteJSONFieldread 2026-09-27

    Build a JSON field from the columns of the incoming rows

  • tWriteXMLFieldread 2026-09-27

    Build an XML structure from the input columns and write it into a field

  • tWriteXMLFieldsread 2026-09-27

    Serialise the columns of each row into one XML string or byte array column

Trifacta (2)
  • Nest Transformread 2026-09-27

    Build an Object or Array column from the names and values of several columns

  • Nest columnsread 2026-09-27

    Convert columns into an Object or Array column

VisiData (1)
  • contract-source-colsread 2026-09-23

    From Columns Sheet, contract selected columns into one grouped column

Background

In DuckDB, a STRUCT is typically used to nest multiple columns into a single column, and the nested column can be of any type.[1] In DuckDB, struct_pack(name := expr, ...) or the notation {'name': expr, ...} builds a STRUCT from expressions.[1] A JavaScript Object Notation (JSON) object is a pair of curly brackets around zero or more name/value pairs, each name being a string.[2]

When to use it

A DuckDB LIST holds values that must all have the same underlying type, while each entry of a STRUCT may have a different type.[3][4] Pack into a struct when the columns differ in type and their names matter; pack into a list only when the columns hold the same kind of thing, such as several phone numbers. (Meridian's view) Every row of a DuckDB STRUCT column must have the same keys, and the names of the entries are part of the schema.[1]

The method

DuckDB builds a LIST with list_value(expr, ...) or the equivalent bracket notation [expr, ...].[3] DuckDB's json_object creates a JSON object from key and value pairs given as arguments, and json_array creates a JSON array from its arguments.[5] A DuckDB TUPLE is a struct whose entries have no names and are referenced by position, created with the row function.[1]

Compared with the alternatives

To build a list from the values of a table column across rows, DuckDB uses the list aggregate function.[3] DuckDB's json_group_array and json_group_object return a JSON array or object built from all the values in an aggregation.[6] Pack when each row should keep its own nested value; aggregate when many rows should collapse into one. (Meridian's view)