Acquire / file · All operations

Read a whole file as one value

A variant of Read another file format

Read each file unparsed into a single text or binary value, one row per file.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Read a whole file as one text value and one binary value, a row for each file

It reads the orders table the proofs share. Create it first:

SQL
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);

Then run the statement:

SQL
COPY (SELECT (('x,y' || chr(10) || '1,2' || chr(10)))::BLOB) TO '<your_scratch_dir>/whole_a.txt' (FORMAT blob);
COPY (SELECT (('left' || chr(10) || chr(10) || 'right'))::BLOB) TO '<your_scratch_dir>/whole_b.txt' (FORMAT blob);
COPY orders TO '<your_scratch_dir>/whole_orders.parquet' (FORMAT parquet);
CREATE TEMP TABLE as_text AS SELECT replace(filename, '<your_scratch_dir>/', '') AS file, content, size FROM read_text('<your_scratch_dir>/whole_*.txt');
CREATE TEMP TABLE as_binary AS SELECT replace(filename, '<your_scratch_dir>/', '') AS file, content, size FROM read_blob('<your_scratch_dir>/whole_orders.parquet');
SELECT file, size FROM as_text ORDER BY file; SELECT file, size FROM as_binary;
SELECT CASE WHEN ((SELECT count(*) = 2 AND list(content ORDER BY file) = ['x,y' || chr(10) || '1,2' || chr(10), 'left' || chr(10) || chr(10) || 'right'] AND list(size ORDER BY file) = [8, 11] FROM as_text) AND (SELECT count(*) = 1 AND typeof(any_value(content)) = 'BLOB' AND any_value(content)[1:4] = 'PAR1'::BLOB AND any_value(size) = octet_length(any_value(content)) FROM as_binary)) THEN 'as expected' ELSE error('the text files were not read as one unparsed value each, or the Parquet file as one binary value') END AS verdict;

Replace <your_scratch_dir> with a directory of your own that you can write to before running the statement.

  • The statement writes whole_a.txt, whole_b.txt and whole_orders.parquet itself, in <your_scratch_dir>.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the text files were not read as one unparsed value each, or the Parquet file as one binary value”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Blob Inputread 2026-09-27

    Read whole files such as images or media as binary objects, one file or a list of files

Apache Hop (1)
Azure Data Factory (1)
DuckDB (2)
  • read_blobread 2026-09-27

    read_blob table function

  • read_textread 2026-09-27

    read_text table function

Pentaho Data Integration (1)
Talend (1)
  • tFileInputRawread 2026-09-27

    Read the whole content of a file into a single output column

Background

In DuckDB, read_text and read_blob read each file into a single row, whose content field holds the entire file as text (VARCHAR) or binary (BLOB).[1] DuckDB's read_text and read_blob accept a filename, a list of filenames or a glob pattern, and return metadata such as the file size and last modified time.[1]

When to use it

DuckDB's read_text first validates a file's content as UTF-8, and on a file with invalid UTF-8 it throws an error suggesting read_blob instead.[1] In DuckDB, the maximum file size allowed for read_text and for read_blob is 3.9 GiB.[1] Reach for a whole-file read when the file itself is the unit: a document to search, a small file to parse with string functions, or a binary to hash or pass on unchanged. (Meridian's view)

DuckDB's read_text and read_blob use projection pushdown, so a query that omits the content column gets file sizes without DuckDB reading the file data.[1] To list a folder's files and sizes, select the filename and size columns and leave the content column out. (Meridian's view) When the underlying file system cannot provide a metadata value such as a timestamp, DuckDB sets that cell to NULL instead.[1]

The method

In DuckDB, the tables returned by read_text and read_blob have the same schema: filename, content, size and last_modified.[1] DuckDB's read_text also supports reading from pipes, such as /dev/stdin.[1]

Sources

  1. [1]Directly Reading Files – DuckDBdocumentation