Publish / database · All operations

Write to a database

Write or bulk load to a database, warehouse or lake.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

arcform has ducklake_publish for DuckLake.

Shape of the result
the table sent out, with nothing new to look at
Level
needs a DuckDB extension

Variants

Append rows to a table

Insert the rows into a database or warehouse table, leaving the rows already there.

Export a whole database

Write every table of a database out to a directory of files.

Create or drop the target

Create or remove the catalog, schema, table or collection that the rows are written into.

Write to a non-relational store

Write rows as documents, keys, graph nodes, search entries or vectors to a NoSQL, search, graph or vector database.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

Alteryx Designer (3)
Apache DataFusion (15)
Apache Hop (27)
  • Beam BigQuery Outputread 2026-09-27

    Write rows to a Google BigQuery table on the Beam engine

  • Beam Bigtable outputread 2026-09-27

    Write rows to a Google Cloud Bigtable table on the Beam engine

  • Bulk load into MSSQLread 2026-09-27

    Bulk load an existing text file into a SQL Server table

  • Bulk load into MySQLread 2026-09-27

    Bulk load a text file into a MySQL table with LOAD DATA

  • Cassandra Outputread 2026-09-27

    Write rows to a Cassandra table

  • CrateDB Bulk Loaderread 2026-09-27

    Bulk load rows into CrateDB through its HTTP endpoint or COPY

  • Deleteread 2026-09-27

    Delete database rows whose keys match each input row

  • Doris Bulk Loaderread 2026-09-27

    Bulk load rows into Apache Doris through stream load

  • LDAP Outputread 2026-09-27

    Insert, update or delete entries in an LDAP directory

  • Bulk load rows into SQL Server through the JDBC bulk copy API

  • Write rows to a table in a Microsoft Access database file

  • MonetDB Bulk Loaderread 2026-09-27

    Bulk load rows into MonetDB

  • MongoDB Outputread 2026-09-27

    Insert, update or upsert documents in a MongoDB collection

  • MongoDB deleteread 2026-09-27

    Delete documents from a MongoDB collection

  • MySQL bulk loaderread 2026-09-27

    Bulk load rows into MySQL through a FIFO file and LOAD DATA

  • Neo4j graph outputread 2026-09-27

    Map fields onto a graph model and write nodes and relationships to Neo4j

  • Neo4j importread 2026-09-27

    Run the Neo4j bulk import command over prepared CSV files

  • Neo4j outputread 2026-09-27

    Create or merge one or two nodes and a relationship in Neo4j per row

  • Oracle Bulk Loaderread 2026-09-27

    Bulk load rows into Oracle through SQL*Loader

  • PostgreSQL bulk loaderread 2026-09-27

    Bulk load rows into PostgreSQL through COPY FROM STDIN

  • Redis Outputread 2026-09-27

    Write string, hash, set or list values to Redis

  • Redshift Bulk Loaderread 2026-09-27

    Bulk load rows into Amazon Redshift through COPY from S3

  • Snowflake bulk loaderread 2026-09-27

    Bulk load rows into Snowflake through staged files and COPY

  • Table Outputread 2026-09-27

    Insert rows into a database table, with batching and partitioned table names

  • Teradata Bulk Loaderread 2026-09-27

    Bulk load rows into Teradata with the fastload utility

  • Vertica Bulk Loaderread 2026-09-27

    Bulk load rows into Vertica through a COPY stream

  • pgvector upsertread 2026-09-27

    Upsert text chunks and their embeddings into a PostgreSQL pgvector table

Apache NiFi (7)
  • DatabaseRecordSinkread 2026-09-27

    Record sink: write records through a database connection

  • 17 processors, MongoDB, DynamoDB, Elasticsearch, Cosmos DB, Data Explorer, BigQuery, Redis, Splunk; incl. deletes

  • PutDatabaseRecordread 2026-09-27

    Translate incoming records to SQL statements and execute them as a single transaction

  • PutDistributedMapCacheread 2026-09-27

    Put FlowFile content into a map cache under a key computed from attributes

  • PutRecordread 2026-09-27

    Send records to whichever record sink service is configured

  • PutSQLread 2026-09-27

    Execute a SQL update or insert held in FlowFile content, with parameters from attributes

  • UpdateDatabaseTableread 2026-09-27

    Generate the table changes a database needs to accept the incoming flat records

Apache SeaTunnel (1)
  • 80 connectors, one per system: Jdbc, Kafka, S3File, LocalFile, Doris, Iceberg, Console, Assert, …

AWS Glue (1)
Azure Data Factory (5)
Bruin (8)
  • DDLread 2026-09-27

    Create the table once from the column definitions in the asset, with no query

  • Load the dataframe, Arrow table or dict rows a Python function returns into a destination table

  • appendread 2026-09-27

    Insert the query result into the table without touching existing rows

  • create+replaceread 2026-09-27

    Create the table if absent and overwrite its contents with the query result on every run

  • 27 destination rows in the strategy table: bigquery, snowflake, postgres, duckdb, clickhouse, …

  • tableread 2026-09-27

    Materialize the asset query result as a table

  • truncate+insertread 2026-09-27

    Truncate the table, keeping its structure, and insert the full query result

  • viewread 2026-09-27

    Materialize the asset query as a view

dbt-utils (4)
  • appendread 2026-09-27

    Insert selected rows into the target without updating, deleting or checking duplicates

  • materialized_viewread 2026-09-27

    Create and maintain a materialized view that the platform refreshes

  • tableread 2026-09-27

    Rebuild the model as a table on each run via create table as

  • viewread 2026-09-27

    Rebuild the model as a view on each run via create view as

DuckDB (1)
Duckle (16)
EasyMorph (5)
  • Bulk load the table into a database table, on databases that support it (paid editions only)

  • Delete database rowsread 2026-09-27

    Delete the rows of a database table that satisfy query conditions

  • Delete database rows whose keys match, or do not match, keys in an EasyMorph table

  • Export to databaseread 2026-09-27

    Insert the table's rows into a database table

  • Update database tableread 2026-09-27

    Update rows of a database table with values from the EasyMorph table (paid editions only)

KNIME Analytics Platform (9)
  • DB Loaderread 2026-09-27

    Bulk load a table into an existing database table using the database's native loader

  • Delete database rows whose key columns match rows of the input table

  • DB Row Deleterread 2026-09-27

    Delete rows of a database table that match configured filter criteria

  • DB Row Inserterread 2026-09-27

    Insert input-table rows into a database table

  • DB Table Deleterread 2026-09-27

    Drop a table from the database

  • Create a new database table from a manual definition or an input table's spec

  • DB Updaterread 2026-09-27

    Update database rows identified by key columns with values from the input table

  • DB Writerread 2026-09-27

    Write a table to a database table, creating the table when it does not exist

  • DB Writer (DB Data)read 2026-09-27

    Materialise the result of a DB query as a new table inside the database

Pentaho Data Integration (26)
Polars (1)
RapidMiner Studio (16)
SQLMesh (5)
  • FULLread 2026-09-27

    Rewrite the whole table on every model evaluation

  • INCREMENTAL_UNMANAGEDread 2026-09-27

    Run the query on schedule and append whatever it returns to the table

  • MANAGEDread 2026-09-27

    Create a table whose data the database engine itself keeps up to date

  • VIEWread 2026-09-27

    Create or replace a non-materialized view; the default kind

  • VIEW (materialized)read 2026-09-27

    VIEW with materialized true, on engines that support materialized views

Tableau Prep (1)
  • Database tableread 2026-09-27

    Write to an external database table: append, create or replace, with SQL before and after

Talend (10)
  • 68 components writing bulk files or running bulk loaders: tMysqlOutputBulk, tOracleBulkExec, tTeradataFastLoad

  • 63 components, one per database, inserting, updating or deleting rows: tMysqlOutput, tOracleOutput, tSnowflakeOutput

  • tHBaseDeleteRowsread 2026-09-27

    Delete rows from an HBase table by row key

  • tLDAPRenameEntryread 2026-09-27

    Rename one or more entries in an LDAP directory

  • tMatchIndexread 2026-09-27

    Index a clean, deduplicated data set in Elasticsearch for continuous matching

  • tNeo4jImportToolread 2026-09-27

    Create a Neo4j database and bulk import CSV files into it with the Neo4j Import Tool

  • tNeo4jv4ImportToolread 2026-09-27

    Create a Neo4j 4.x database and bulk import CSV files into it with the Neo4j Import Tool

  • tPineconeClientread 2026-09-27

    Upsert, update or delete records in a Pinecone index namespace, or query and fetch them

  • Call Sqoop to transfer data from HDFS to a relational database

  • tSynonymOutputread 2026-09-27

    Create a Lucene index and feed it with entries and their synonyms

Background

A data manipulation language (DML) is used for adding (inserting), deleting and modifying (updating) data in a database.[1] In Structured Query Language (SQL), DML statements are organised by their first word, a verb such as INSERT INTO, UPDATE or DELETE FROM.[1] In extract, transform, load (ETL), the load phase puts the data into its end target, which can be any data store, from a flat file to a data warehouse.[2]

When to use it

Some data warehouses may overwrite existing information with cumulative information, while others may add new data in a historical form at regular intervals.[2] More complex ETL systems can keep a history and audit trail of all changes to the data loaded into the data warehouse.[2] In ETL, when to replace and when to append are strategic design choices that depend on the time available and the business needs.[2] In Bruin, the append strategy does not deduplicate, so running the same time window twice inserts the rows twice.[3] Decide before the first run whether a run adds to its target or replaces it, and what a second run over the same data should leave behind. (Meridian's view)

When the load phase writes to a database, the constraints in the database schema, such as uniqueness, referential integrity and mandatory fields, apply, as do triggers activated on load.[2] In SQLite, the datatype of a value is associated with the value itself, not with the column it is stored in.[4] SQLite has provided STRICT tables, which enforce types rigidly, since version 3.37.0 (2021-11-27).[4] Check what a target database enforces before relying on it to reject a bad row. (Meridian's view)

In PostgreSQL, a row can be reliably addressed only through a primary key, because SQL does not in general provide a unique identifier for rows.[5] Give a table a key before writing to it in any way other than appending, so that a change can name the rows it means. (Meridian's view)

The method

In SQL, statements that create or change tables and other database objects, rather than the data in them, belong to a separate data definition language (DDL).[1] In DuckDB, INSERT INTO adds new rows to a table, either rows given as value expressions or the rows a query returns.[6] In DuckDB, INSERT INTO ... BY NAME matches the column names of a SELECT query against the table's, so the columns may come in a different order or some may be missing.[6] In DuckDB, a column left out of an insert is filled with its declared default value, or NULL if it has none.[6]

In DuckDB, the ATTACH statement adds a database file to the catalogue, where it can be read from and written to.[7] In DuckDB, ATTACH definitions are not kept between sessions, so each new session has to attach its databases again.[7] DuckDB's SQLite extension can create SQLite database files, create tables and ingest data into SQLite with standard SQL queries.[8]

Compared with the alternatives

DuckDB's EXPORT DATABASE command exports the contents of a database to a directory, and its IMPORT DATABASE command reads them back.[9]