Acquire / file · All operations

Read a columnar file

Read Parquet, ORC, Avro, Arrow, Iceberg, Delta.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

Shape of the result
a new table, from no input table
Level
one clause of SQL

The DuckDB step

Read Parquet

It reads the orders table the proofs share. Create it first:

SQL
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);

Then run the statement:

SQL
COPY orders TO '<your_scratch_dir>/orders.parquet' (FORMAT parquet);
CREATE TEMP TABLE result AS
SELECT count(*) AS n, sum(amount) AS total FROM read_parquet('<your_scratch_dir>/orders.parquet');
SELECT * FROM result;
SELECT CASE WHEN ((SELECT n = 10 AND total = 1900.74 FROM result)) THEN 'as expected' ELSE error('the Parquet file did not read back as 10 rows totalling 1900.74') END AS verdict;

Replace <your_scratch_dir> with a directory of your own that you can write to before running the statement.

  • The statement writes orders.parquet itself, in <your_scratch_dir>.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the Parquet file did not read back as 10 rows totalling 1900.74”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

Variants

Read another columnar format

Read a table from a DuckDB database file, a Vortex file or another columnar store.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

Apache DataFusion (3)
  • Arrowread 2026-09-27

    Read Arrow IPC files as a table

  • Avroread 2026-09-27

    Read Avro files as a table

  • Parquetread 2026-09-27

    Read Parquet files as a table, including nested types

Apache Hop (4)
  • Avro file inputread 2026-09-27

    Read Avro records from one or more files

  • Data stream inputread 2026-09-27

    Read rows from an Arrow file, random-access file or Flight data stream

  • Parquet file inputread 2026-09-27

    Read fields from Apache Parquet files

  • Spark lake table inputread 2026-09-27

    Read Delta Lake or Iceberg tables on native Spark, with time travel and projection

Apache NiFi (1)
  • AvroReaderread 2026-09-27

    Read Avro data into records, with an embedded or an external schema

Apache SeaTunnel (1)
  • Avro formatread 2026-09-27

    Read and write Kafka messages in Avro

AWS Glue (6)
  • Avro (read)read 2026-09-27

    Read Avro data from Amazon S3 and other supported connection types

  • Delta Lake (read)read 2026-09-27

    Read Delta Lake tables

  • Hudi (read)read 2026-09-27

    Read Hudi tables

  • Iceberg (read)read 2026-09-27

    Read Iceberg tables

  • ORC (read)read 2026-09-27

    Read ORC data from Amazon S3 and other supported connection types

  • Parquet (read)read 2026-09-27

    Read Parquet data from Amazon S3 and other supported connection types

Azure Data Factory (4)
DuckDB (6)
  • arrow_scanread 2026-09-27

    arrow_scan table function

  • arrow_scan_dumbread 2026-09-27

    arrow_scan_dumb table function

  • parquet_scanread 2026-09-27

    parquet_scan table function

  • read_duckdbread 2026-09-27

    read_duckdb table function

  • read_parquetread 2026-09-27

    read_parquet table function

  • seq_scanread 2026-09-27

    seq_scan table function

Duckle (5)
  • Apache Icebergread 2026-09-25

    read an Iceberg table with iceberg_scan or an attached REST catalog

  • Avroread 2026-09-25

    read Avro container files; the run path decodes in Rust, inspect uses read_avro

  • Delta Lakeread 2026-09-25

    read a Delta Lake table with delta_scan

  • Parquetread 2026-09-25

    read Parquet files with read_parquet

  • Vortexread 2026-09-25

    read Vortex columnar files, bridged through Parquet

EasyMorph (1)
KNIME Analytics Platform (1)
Pentaho Data Integration (4)
  • Avro Inputread 2026-09-27

    Decode binary or JSON Avro data from files or incoming fields into fields

  • Avro input (deprecated)read 2026-09-27

    Deprecated Avro reader, replaced by Avro Input

  • ORC Inputread 2026-09-27

    Read fields from ORC files

  • Parquet Inputread 2026-09-27

    Decode Parquet files and extract fields

Polars (10)
  • read_avroread 2026-09-27

    Read an Avro file into a frame

  • read_deltaread 2026-09-27

    Read a Delta Lake table into a frame

  • read_ipcread 2026-09-27

    Read an Arrow IPC (Feather v2) file into a frame

  • read_ipc_streamread 2026-09-27

    Read an Arrow IPC record-batch stream into a frame

  • read_parquetread 2026-09-27

    Read a Parquet file into a frame

  • scan_deltaread 2026-09-27

    Lazily read a Delta Lake table

  • scan_icebergread 2026-09-27

    Lazily read an Apache Iceberg table

  • scan_ipcread 2026-09-27

    Lazily read one or many Arrow IPC files, by glob pattern

  • scan_parquetread 2026-09-27

    Lazily read local or cloud-hosted Parquet files

  • scan_pyarrow_datasetread 2026-09-27

    Lazily read a pyarrow dataset

Power Query M (5)
  • Arrow.Documentread 2026-09-27

    Read an Apache Arrow document into a table

  • Arrow.FromBinaryread 2026-09-27

    Read an Apache Arrow IPC stream, such as an HTTP response body, into a table

  • DeltaLake.Metadataread 2026-09-27

    Read the transaction log entries of a Delta Lake table

  • DeltaLake.Tableread 2026-09-27

    Read the contents of a Delta Lake table from a directory listing

  • Parquet.Documentread 2026-09-27

    Read a Parquet file binary into a table; named on the Parquet connector page only

Talend (5)
VisiData (3)
Yggdrasil Decision Forests (1)
  • avro: (read)read 2026-09-27

    Read Avro files by typed path; recommended for large datasets; no Avro writer is registered

Background

In a column-oriented format, each column of a table is stored one after the other, so values in the same column sit close together.[1] Apache Parquet, Optimized Row Columnar (ORC) and Apache Arrow are examples of column-oriented formats.[1][2]

When to use it

Row-oriented formats are more commonly used in online transaction processing, and column-oriented formats in online analytical processing.[1] Row-oriented formats benefit from fast insertion of a new row, and column-oriented formats from fast insertion of a new column.[1] Reach for a columnar file when the work reads a few columns across many rows, as totals, counts and profiles do, and keep a row format for work that writes or fetches whole records one at a time. (Meridian's view)

The method

Tabular data is two-dimensional, but computers represent data in a linear memory model, so storing a table means mapping its rows and columns into one dimension.[1] A row-oriented layout instead stores each row of a table one after the other, so values in the same row sit close together.[1] Column-oriented formats benefit from a smaller compressed size, the result of higher homogeneity within a column than across rows.[1] A column-oriented format can also give a data type a dedicated encoding: 128 Boolean values take 128 bits as a bitmap, against 128 bytes at one byte each in a row-oriented format.[1] Column-oriented formats benefit from fast analytics operations, because they can use single instruction, multiple data (SIMD) instructions.[1]

Compared with the alternatives

Comma-separated values (CSV) files are a predominant example of a row-oriented format.[1] Because both orientations represent the same data, a row-oriented dataset can be converted to a column-oriented one and back, at the expense of compute.[1] Advanced query engines often convert from one orientation to the other as part of their execution, to use each orientation's advantages.[1]

Sources

  1. [1]Data orientation - Wikipediasecondary source
  2. [2]Apache ORC - Wikipediasecondary source

Used in

4 Protocols use this operation: