Publish / file · All operations

Write a machine-learning data file

A variant of Write another file format

Write a table to a format machine-learning libraries read, such as ARFF, TFRecord or a NumPy array.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

KNIME Analytics Platform (1)
  • ARFF Writerread 2026-09-27

    Write a table to a Weka ARFF file

Pentaho Data Integration (1)
  • ARFF outputread 2026-09-27

    Write rows to a Weka ARFF file

RapidMiner Studio (1)
  • Write ARFFread 2026-09-27

    Write a table to a Weka ARFF file

Talend (1)
VisiData (1)
Yggdrasil Decision Forests (2)
  • convert_datasetread 2026-09-27

    Rewrite a dataset from one typed-path format to another using its dataspec

  • ydf.util.write_tf_recordread 2026-09-27

    Write an in-memory dataset to a TFRecord file of tf.Example rows

Background

In RapidMiner, the Write ARFF operator writes data as ARFF (Attribute-Relation File Format) files, known from the machine learning library Weka.[1] The TFRecord format is a simple format for storing a sequence of binary records.[2] The .npy format is the standard binary file format in NumPy for persisting a single arbitrary NumPy array on disk.[3]

When to use it

For TFRecord data, TensorFlow's rule of thumb is to shard across at least 10 times as many files as there will be hosts reading, each file at least 10 MB and ideally 100 MB or more.[2]

In ARFF, a missing value is represented by a single question mark.[4] In a sparse ARFF file, omitted values are 0 rather than missing, so an unknown value must be written explicitly as a question mark.[4] When writing ARFF, turn every NULL into a question mark rather than an empty field or a zero. (Meridian's view)

The .npy format does not completely preserve arbitrary subclasses of numpy.ndarray: only the array data is written, and a regular numpy.ndarray is created on reading.[3]

The method

An ARFF file is an ASCII text file with a header section, holding the relation's name and its attributes with their types, followed by a data section.[4] In ARFF, the order in which the attributes are declared gives their column position in the data section.[4] In the ARFF data section, each instance is on a single line, and its values are delimited by commas or tabs.[4]

A TFRecord file contains a sequence of records and can only be read sequentially.[2] Each TFRecord record contains a byte-string payload, plus the payload's length and hashes for integrity checking.[2] The tf.train.Example message is a flexible message type that represents a mapping from strings to values, though TFRecord files do not require it.[2]

A .npy file stores the shape and data type (dtype) information needed to reconstruct the array correctly, even on a machine with a different architecture.[3] The .npz format is a zip file containing several .npy files, one for each array.[3] The .npy format is straightforward to reverse engineer, and allows memory-mapping of the data.[3]

Compared with the alternatives

Where an ARFF file is an ASCII text file, the .npy format represents data in its native binary form.[4][3]

Sources

  1. [1]Write ARFF - Rapidminer Documentationdocumentation
  2. [2]TFRecord and tf.train.Example | TensorFlow Coredocumentation
  3. [3]numpy.lib.format — NumPy v2.5 Manualdocumentation
  4. [4]Arff stable - Weka Wikidocumentation