Publish / file · All operations

Write an ORC file

A variant of Write a columnar file

Write rows to a file in ORC format.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

AWS Glue (1)
  • ORC (write)read 2026-09-27

    Write ORC files and folders to Amazon S3

Azure Data Factory (1)
  • ORC format (sink)read 2026-09-27

    Write ORC files in copy activity and mapping data flow

Pentaho Data Integration (1)
  • ORC Outputread 2026-09-27

    Write rows to a file in ORC format

Talend (1)

Background

Apache ORC (Optimized Row Columnar) is a free and open-source column-oriented data storage format.[1] ORC is a self-describing, type-aware columnar file format designed for Hadoop workloads.[2] ORC files were created in January 2013 as part of an initiative to speed up Apache Hive and improve the storage efficiency of data stored in Apache Hadoop.[2]

When to use it

ORC is optimised for large streaming reads, with integrated support for finding required rows quickly.[2] ORC supports the complete set of types in Hive, including the complex types structs, lists, maps and unions.[2] ORC is used by data processing frameworks including Apache Spark, Apache Hive, Apache Flink and Apache Hadoop.[1] Facebook uses ORC to save tens of petabytes in its data warehouse.[2] Write ORC when the files will be read in a Hive-centred stack, and check that every other reader you rely on supports it before choosing it over Parquet. (Meridian's view)

The method

ORC files are divided into stripes that are roughly 64MB by default, and the stripes in a file are independent of each other.[2] Within each ORC stripe, the columns are separated from each other, so a reader can read just the columns it requires.[2] Because ORC files are type-aware, the writer chooses the most appropriate encoding for the type and builds an internal index as the file is written.[2] ORC's lightweight compression techniques include dictionary encoding, bit packing, delta encoding and run length encoding, and ORC can apply zlib or Snappy compression on top.[3] ORC files include lightweight indexes holding the minimum and maximum values for each column in each set of 10,000 rows and in the entire file.[3] In ORC, predicate pushdown uses the indexes to determine which stripes in a file need to be read for a query, and the row indexes can narrow the search to a set of 10,000 rows.[2]

ORC stores its top-level index at the end of the file, because the Hadoop distributed file system (HDFS) does not support changing the data in a file after it is written.[3][4] ORC stores its metadata using Protocol Buffers, which can add new fields without breaking readers.[3]

Compared with the alternatives

The ORC format was announced by Hortonworks in collaboration with Facebook in February 2013, and the Apache Parquet format a calendar month later, developed by Cloudera and Twitter.[1]

Sources

  1. [1]Apache ORC - Wikipediasecondary source
  2. [2]Backgrounddocumentation
  3. [3]ORC Specification v1standard
  4. [4]Data lake - Wikipediasecondary source