# Read a file's schema and metadata

Shape / inspect

A variant of [List columns and types](https://meridian.online/operations/list-columns-and-types).

Read a data file's schema, row groups, statistics, header or detected format without loading its data.

Page: <https://meridian.online/operations/list-columns-and-types/read-a-files-schema-and-metadata>

## Status

Runs as a SQL step.

arcform carries it as a `sql:` step today.

## The DuckDB step

read a file's schema, row groups, statistics, header and detected format without loading its rows

It reads the `orders` table the proofs share. Create it first:

```sql
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);
```

Then run the statement:

```sql
-- 5000 rows written as three row groups of at most 2048 rows
COPY (SELECT i AS id, 'row ' || i AS label FROM range(5000) t(i)) TO '<your_scratch_dir>/inspect_many.parquet' (FORMAT parquet, ROW_GROUP_SIZE 2048);
COPY (SELECT order_id, customer, order_date, amount FROM orders) TO '<your_scratch_dir>/inspect_orders.csv' (HEADER);
CREATE TEMP TABLE file_schema AS SELECT name, duckdb_type FROM parquet_schema('<your_scratch_dir>/inspect_many.parquet') WHERE num_children IS NULL;
CREATE TEMP TABLE file_info AS SELECT num_rows, num_row_groups FROM parquet_file_metadata('<your_scratch_dir>/inspect_many.parquet');
CREATE TEMP TABLE row_groups AS
SELECT row_group_id, stats_min::BIGINT AS id_min, stats_max::BIGINT AS id_max, num_values
FROM parquet_metadata('<your_scratch_dir>/inspect_many.parquet') WHERE path_in_schema = 'id';
CREATE TEMP TABLE csv_format AS
SELECT "Delimiter" AS delimiter, "HasHeader" AS has_header, [c.name FOR c IN "Columns"] AS column_names, [c.type FOR c IN "Columns"] AS column_types
FROM sniff_csv('<your_scratch_dir>/inspect_orders.csv');
SELECT * FROM file_schema;
SELECT * FROM file_info;
SELECT * FROM row_groups ORDER BY row_group_id;
SELECT * FROM csv_format;
SELECT CASE WHEN ((SELECT list((name, duckdb_type) ORDER BY name) FROM file_schema) = [('id', 'BIGINT'), ('label', 'VARCHAR')] AND (SELECT (num_rows, num_row_groups) FROM file_info) = (5000, 3) AND (SELECT list((id_min, id_max, num_values) ORDER BY row_group_id) FROM row_groups) = [(0, 2047, 2048), (2048, 4095, 2048), (4096, 4999, 904)] AND (SELECT (delimiter, has_header) FROM csv_format) = (',', true) AND (SELECT column_names FROM csv_format) = ['order_id', 'customer', 'order_date', 'amount'] AND (SELECT column_types FROM csv_format) = ['BIGINT', 'VARCHAR', 'DATE', 'DOUBLE']) THEN 'as expected' ELSE error('the Parquet footer was not 3 row groups over 5000 rows with their id ranges, or the CSV was not detected as comma-separated with a header and four typed columns') END AS verdict;
```

Replace `<your_scratch_dir>` with a directory of your own that you can write to before running the statement.

The statement writes `inspect_many.parquet` and `inspect_orders.csv` itself, in `<your_scratch_dir>`.

It needs nothing installed beyond DuckDB itself.

## What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

- Apache DataFusion: [`parquet_metadata`](https://datafusion.apache.org/user-guide/cli/functions.html) (read 2026-09-27)
- DuckDB: `parquet_bloom_probe` (read 2026-09-27)
- DuckDB: `parquet_file_metadata` (read 2026-09-27)
- DuckDB: `parquet_full_metadata` (read 2026-09-27)
- DuckDB: `parquet_kv_metadata` (read 2026-09-27)
- DuckDB: `parquet_metadata` (read 2026-09-27)
- DuckDB: `parquet_schema` (read 2026-09-27)
- Apache Hop: [`File metadata`](https://hop.apache.org/manual/latest/pipeline/transforms/filemetadata.html) (read 2026-09-27)
- Apache NiFi: [`ExtractAvroMetadata`](https://nifi.apache.org/components/org.apache.nifi.processors.avro.ExtractAvroMetadata/) (read 2026-09-27)
- Apache NiFi: [`IdentifyMimeType`](https://nifi.apache.org/components/org.apache.nifi.processors.standard.IdentifyMimeType/) (read 2026-09-27)
- Pentaho Data Integration: [`Read metadata from Copybook`](https://docs.pentaho.com/pdia-data-integration/9.3-data-integration/pdi-transformation-steps-reference-overview) (read 2026-09-27)
- Polars: [`read_ipc_schema`](https://docs.pola.rs/api/python/stable/reference/io.html) (read 2026-09-27)
- Polars: [`read_parquet_metadata`](https://docs.pola.rs/api/python/stable/reference/io.html) (read 2026-09-27)
- Polars: [`read_parquet_schema`](https://docs.pola.rs/api/python/stable/reference/io.html) (read 2026-09-27)
- VisiData: [`dive-metadata`](https://github.com/saulpw/visidata/blob/7609212ec2febafa9caba7183cbebfaf49627909/visidata/loaders/hdf5.py#L87) (read 2026-09-23)