Acquire / generate · All operations

Generate random or synthetic data

A variant of Generate rows

Generate rows or columns of random, fake or simulated values for testing.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Generate seeded random numbers, fake names and a normal sample

SQL
SELECT setseed(0.42);
CREATE TEMP TABLE first_run AS
SELECT i, random() AS u, (['Ann', 'Bo', 'Cy', 'Di'])[1 + (random() * 4)::INTEGER % 4] AS first_name
FROM range(1000) t(i);
SELECT setseed(0.42);
CREATE TEMP TABLE second_run AS
SELECT i, random() AS u, (['Ann', 'Bo', 'Cy', 'Di'])[1 + (random() * 4)::INTEGER % 4] AS first_name
FROM range(1000) t(i);
-- simulated values: a normal sample by the Box-Muller transform from two uniforms
SELECT setseed(0.42);
CREATE TEMP TABLE normal AS
SELECT sqrt(-2 * ln(1 - random())) * cos(2 * pi() * random()) AS z FROM range(2000);
SELECT * FROM first_run LIMIT 3;
SELECT CASE WHEN ((SELECT count(*) = 1000 AND min(u) >= 0 AND max(u) < 1 AND count(DISTINCT u) = 1000 AND count(DISTINCT first_name) = 4 FROM first_run) AND (SELECT count(*) FROM first_run a JOIN second_run b USING (i) WHERE a.u = b.u AND a.first_name = b.first_name) = 1000 AND (SELECT abs(avg(z)) < 0.1 AND stddev_pop(z) BETWEEN 0.9 AND 1.1 FROM normal)) THEN 'as expected' ELSE error('the seeded run did not repeat exactly, or the uniforms, the four fake names or the normal sample were out of range') END AS verdict;
  • The statement builds its own data.
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the seeded run did not repeat exactly, or the uniforms, the four fake names or the normal sample were out of range”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Synthetic Inputread 2026-09-27

    Generate synthetic rows with generative AI to build a workflow before real data exists

Apache Hop (2)
  • Fake dataread 2026-09-27

    Generate fake names, addresses and other values with the Java Faker library

  • Generate Random Valueread 2026-09-27

    Add random numbers, strings, UUIDs or hashes as new fields

Apache NiFi (2)
  • GenerateFlowFileread 2026-09-27

    Create FlowFiles with random data or custom content

  • GenerateRecordread 2026-09-27

    Create records with random values for the fields of a schema

KNIME Analytics Platform (2)
  • Data Generatorread 2026-09-27

    Generate random clustered numeric data with a share of noise, normalised to 0..1

  • KNIME Data Generation (extension)read 2026-09-27

    Extension outside the base install: 21 current nodes: Beta Distributed Assigner; Conditional Label Assigner; Counter Generation, …

mlpack (2)
  • mlpack_gmm_generateread 2026-09-27

    Sample new points at random from a trained Gaussian mixture model

  • mlpack_hmm_generateread 2026-09-27

    Generate a random observation sequence from a trained hidden Markov model

Pentaho Data Integration (2)
Polars (1)
RapidMiner Studio (13)
Talend (1)
  • tRowGeneratorread 2026-09-27

    Generate any number of rows and fields of random values, as test data

Yggdrasil Decision Forests (1)
  • synthetic_datasetread 2026-09-27

    Generate synthetic train, validation and test datasets from an options file

Background

Synthetic data are artificially generated data not produced by real-world events, and are typically created using algorithms.[1] In DuckDB, random() returns a random number x in the range 0.0 <= x < 1.0, and setseed sets the seed that random uses.[2]

When to use it

Synthetic data can protect the privacy and confidentiality of authentic data while still allowing for use in testing systems.[1] Privacy regulations restrict the use of privacy-sensitive personal data for testing, though anonymised production data may be used as representative data for testing.[3] Generating synthetic data that accurately reflects real-world complexity can be challenging, and synthetic data may miss nuances of real data, leaving gaps in test coverage.[3] Alteryx's Synthetic Input tool creates synthetic data with generative AI, so that workflows can be built before there is access to the real data.[4] Use random rows to exercise a pipeline's shape and its edge cases, and keep real or anonymised samples for anything that judges what the data says. (Meridian's view)

A pseudorandom number generator's sequence is completely determined by its seed, so reinitialising the generator with the same seed produces the same sequence of numbers.[5] RapidMiner's Generate Data operator generates an ExampleSet, and using the same value of its local random seed produces the same ExampleSet.[6] Apache Hop's Generate Random Value transform takes a seed for its random numbers, integers and strings.[7] Set a seed whenever generated values feed a test or a published example, so that the next run gives the same rows. (Meridian's view)

Apache NiFi's GenerateRecord processor takes a Null Value Percentage, the probability that a generated value for any nullable field will be null.[8]

The method

Apache Hop's Fake data transform generates fake data with the Java Faker library, and a locale can make the generated data more specific to one location.[9] Apache NiFi's GenerateRecord defines a record schema from a provided schema text, a predefined template such as Person, Order or Sensor, or user-defined dynamic properties.[8]

The Box–Muller transform, introduced by George Edward Pelham Box and Mervin Edgar Muller, generates pairs of independent, standard, normally distributed random numbers from uniformly distributed ones.[10] Box and Muller published A Note on the Generation of Random Normal Deviates in The Annals of Mathematical Statistics in 1958.[10] The Box–Muller transform was developed as a more computationally efficient alternative to the inverse transform sampling method.[10] In privacy-preserving statistical analysis, the idea of original fully synthetic data was created by Donald Rubin in 1993.[1]

Sources

  1. [1]Synthetic data - Wikipediasecondary source
  2. [2]Numeric Functions – DuckDBdocumentation
  3. [3]Test data - Wikipediasecondary source
  4. [4]Synthetic Input Tooldocumentation
  5. [5]Random seed - Wikipediasecondary source
  6. [6]Generate Data - Rapidminer Documentationdocumentation
  7. [7]Generate Random Value | Apache Hopdocumentation
  8. [8]GenerateRecord - Apache NiFidocumentation
  9. [9]Fake data | Apache Hopdocumentation
  10. [10]Box–Muller transform - Wikipediasecondary source