# Hash several columns into one key

Shape / columns

A variant of [Hash or mask a column](https://meridian.online/operations/hash-or-mask-a-column).

Combine several columns into one checksum or surrogate-key column.

Page: <https://meridian.online/operations/hash-or-mask-a-column/hash-columns-into-a-key>

## Status

Runs as a SQL step.

arcform carries it as a `sql:` step today.

## The DuckDB step

hash several columns into one key that does not collide on separators or nulls

```sql
CREATE TEMP TABLE parts AS SELECT * FROM (VALUES
  (1, 'a|b', 'c', 'x'), (2, 'a', 'b|c', 'x'), (3, 'a', NULL, 'x'), (4, NULL, 'a', 'x'), (5, 'a', '', 'x'), (6, 'a|b', 'c', 'x')
) t(id, k1, k2, k3);
-- JSON text of the row keeps each part apart, so a pipe in a value and a null are not mistaken for a separator or an empty string
CREATE TEMP TABLE result AS
SELECT id, md5(to_json(row(k1, k2, k3))) AS row_key FROM parts;
SELECT * FROM result ORDER BY id;
SELECT CASE WHEN ((SELECT count(DISTINCT row_key) FROM result) = 5 AND (SELECT count(*) FROM result WHERE length(row_key) = 32) = 6 AND (SELECT count(DISTINCT row_key) FROM result WHERE id IN (1, 6)) = 1 AND (SELECT row_key FROM result WHERE id = 1) = md5('["a|b","c","x"]')) THEN 'as expected' ELSE error('the rows were not given one key each, equal rows the same key, with a pipe, a null and an empty string kept apart') END AS verdict;
```

The statement builds its own data.

It needs nothing installed beyond DuckDB itself.

## What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

- Bruin: [`bruin.generate_surrogate_key(fields)`](https://getbruin.com/docs/bruin/assets/templating/macros.html) (read 2026-09-27)
- dbt-utils: [`generate_surrogate_key`](https://raw.githubusercontent.com/dbt-labs/dbt-utils/main/README.md) (read 2026-09-27)
- Duckle: [`Row Hash (fingerprint)`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/website/docs/components.html) (read 2026-09-25)
- Apache Hop: [`Add a checksum`](https://hop.apache.org/manual/latest/pipeline/transforms/addchecksum.html) (read 2026-09-27)
- Pentaho Data Integration: [`Add a Checksum`](https://docs.pentaho.com/pdia-data-integration/9.3-data-integration/pdi-transformation-steps-reference-overview) (read 2026-09-27)
- Polars: [`DataFrame.hash_rows`](https://docs.pola.rs/api/python/stable/reference/dataframe/computation.html) (read 2026-09-27)
- SQLMesh: [`@GENERATE_SURROGATE_KEY`](https://github.com/SQLMesh/sqlmesh/blob/23b237cea7218cc8f338b89c14f0957394a0003b/docs/concepts/macros/sqlmesh_macros.md) (read 2026-09-27)
- Talend: [`tAddCRCRow`](https://help.qlik.com/talend/en-US/components/8.0/identification/identification-component) (read 2026-09-27)