Control / log ยท All operations

Log and annotate a run

Logging, audit, metrics, annotations, comments.

Status

In arcform

A feature, not a step

It lives in how a Protocol runs. src/contract.rs in arcform at fc62d6f

Shape of the result
not a data operation
Level
not a step

Variants

Track experiments and runs

Record experiment runs with their parameters and results in a tracking store and read them back.

Keep a replayable command history

Record every command or change of a session so it can be reviewed, undone or replayed.

Set how much is logged

Turn logging, profiling or verbose output on or off for a run.

Count, time and publish run metrics

Increment counters, record gauges and timers, and publish metrics or progress while a run proceeds.

Write a message to the log

Write a message, warning or the passing rows to the run log or console, and pass the data through unchanged.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

Alteryx Designer (5)
  • API Outputread 2026-09-27

    Print the stream to the Results window as quoted comma-delimited text for API callers

  • Commentread 2026-09-27

    Annotate the canvas with a note

  • Explorer Boxread 2026-09-27

    Show a web page, directory or file in a box on the canvas as supporting documentation

  • Messageread 2026-09-27

    Write a message, warning or error to the Results window from an expression

  • Tool Containerread 2026-09-27

    Group tools on the canvas; a container can be collapsed or disabled

Apache DataFusion (6)
  • EXPLAINread 2026-09-27

    Show the logical and physical plan of a statement: tree, indent, pgjson or graphviz

  • EXPLAIN ANALYZEread 2026-09-27

    Run the statement and show its plan with execution metrics

  • explainread 2026-09-27

    DataFrame method: Return the plan of the DataFrame so far

  • list_files_cacheread 2026-09-27

    Table function listing the contents of the listed-files cache

  • metadata_cacheread 2026-09-27

    Table function listing the contents of the file metadata cache

  • statistics_cacheread 2026-09-27

    Table function listing the contents of the file statistics cache

Apache Hop (9)
  • Execution informationread 2026-09-27

    Read or delete execution records from an execution information location

  • Neo4j Get Logging Inforead 2026-09-27

    Read execution logging information stored in a Neo4j logging graph

  • Wait for named transforms to finish and emit their row counts and timings as rows

  • Pipeline Data Proberead 2026-09-27

    Receive rows sampled from other pipelines by a pipeline probe

  • Pipeline loggingread 2026-09-27

    Receive logging records about pipeline executions and their transforms

  • Server Statusread 2026-09-27

    Query the Hop servers named in a field and return their status

  • Workflow loggingread 2026-09-27

    Receive logging records about workflow executions and their actions

  • Write to Logread 2026-09-27

    Write a message to the log at a chosen level

  • Write to logread 2026-09-27

    Write chosen field values and a message to the log

Apache NiFi (10)
  • DebugFlowread 2026-09-27

    Testing aid: trigger chosen framework responses when a FlowFile or timer event arrives

  • Labelread 2026-09-27

    Documentation text placed on the canvas beside parts of a dataflow

  • LogAttributeread 2026-09-27

    Write the attributes of a FlowFile to the log at a chosen level

  • LogMessageread 2026-09-27

    Write a message to the log at a chosen level

  • LoggingRecordSinkread 2026-09-27

    Record sink: write records to the application log

  • MonitorActivityread 2026-09-27

    Signal when the flow has had no data for a set time, and again when activity resumes

  • PutCloudWatchMetricread 2026-09-27

    Publish a metric value or statistic set to Amazon CloudWatch

  • 10 components that publish metrics, status, bulletins and provenance events, or monitor disk and memory

  • UpdateCounterread 2026-09-27

    Increment a named counter, for debugging and basic counting in the flow

  • UpdateGaugeread 2026-09-27

    Record a value for a named gauge for each FlowFile processed

dbt-utils (3)
  • log_inforead 2026-09-27

    Log a timestamped message to the command line

  • pretty_log_formatread 2026-09-27

    Format a message with a timestamp prefix for command-line logging

  • pretty_timeread 2026-09-27

    Return the current timestamp as a formatted string at compile time

DuckDB (10)
  • Profilingread 2026-09-27

    Report where a query spent its time

  • disable_loggingread 2026-09-27

    disable_logging table function

  • disable_profilingread 2026-09-27

    disable_profiling table function

  • duckdb_log_contextsread 2026-09-27

    duckdb_log_contexts table function

  • duckdb_logsread 2026-09-27

    duckdb_logs table function

  • duckdb_logs_parsedread 2026-09-27

    duckdb_logs_parsed table function

  • duckdb_profiling_settingsread 2026-09-27

    duckdb_profiling_settings table function

  • enable_loggingread 2026-09-27

    enable_logging table function

  • enable_profilingread 2026-09-27

    enable_profiling table function

  • truncate_duckdb_logsread 2026-09-27

    truncate_duckdb_logs table function

Duckle (3)
  • Log Messageread 2026-09-25

    write an info log line with the upstream row count, then pass rows through

  • Run Eventsread 2026-09-25

    rows describing the stages that have already failed in this run

  • Warnread 2026-09-25

    write a warning log line, then pass rows through

EasyMorph (3)
  • Generate documentationread 2026-09-27

    Generate HTML documentation of an EasyMorph project

  • Statusread 2026-09-27

    Set a custom execution status message, optionally computed by an expression

  • To-doread 2026-09-27

    Leave a to-do note in the workflow

KNIME Analytics Platform (2)
  • Global Timer Inforead 2026-09-27

    Report node usage and timing collected over the life of the installation

  • Timer Inforead 2026-09-27

    Report execution timing for every node of the workflow

mlpack (2)
  • mlpack_silentread 2026-09-27

    Boolean setting that suppresses the short summaries the functions print, such as Misclassified

  • mlpack_verboseread 2026-09-27

    Boolean setting that prints the input matrices and fitted values while a function runs

OpenRefine (1)
  • History (Undo/Redo)read 2026-09-27

    Keep every data change as a numbered list saved with the project; click an entry to go to that state

Pentaho Data Integration (3)
Polars (3)
RapidMiner Studio (26)
  • Annotateread 2026-09-27

    Add or change key-value annotations on an object

  • Annotations to Dataread 2026-09-27

    Extract an object's annotations into a table

  • Clear Logread 2026-09-27

    Clear a log table created by the Log operator

  • Create Experimentread 2026-09-27

    Create an experiment on an MLflow tracking server

  • Create Runread 2026-09-27

    Create a run inside an existing MLflow experiment

  • Data to Annotationsread 2026-09-27

    Set an object's annotations from the rows of a table

  • Delete Experimentread 2026-09-27

    Delete an experiment on an MLflow tracking server

  • Delete Runread 2026-09-27

    Delete an MLflow run

  • Extract Log Valueread 2026-09-27

    Expose a cell value of a table as a value the Log operator can record

  • Get Experimentsread 2026-09-27

    List the experiments on an MLflow tracking server

  • Get Jobs (AI Hub)read 2026-09-27

    List the jobs executed on an AI Hub server as a table

  • Get Log (AI Hub)read 2026-09-27

    Fetch the execution log of a job on an AI Hub server

  • Get Log (RTSA)read 2026-09-27

    Fetch the contents of a log file from a Real-Time Scoring Agent

  • Get Logs (RTSA)read 2026-09-27

    List the log files of a Real-Time Scoring Agent

  • Get Metrics (AI Hub)read 2026-09-27

    Fetch the current health metrics of an AI Hub server

  • Get Runread 2026-09-27

    Fetch one MLflow run by its run id

  • Get Runsread 2026-09-27

    List the runs of an MLflow experiment

  • Logread 2026-09-27

    Record operator values and parameters to a log table on each execution

  • Log Operator Runtimesread 2026-09-27

    Turn on logging of each operator's runtime for the current process

  • Log Runread 2026-09-27

    Log model evaluation results (metrics, parameters) into an existing MLflow run

  • Log to Dataread 2026-09-27

    Convert a log table into a data table

  • Log to Weightsread 2026-09-27

    Build feature weights from column names recorded in a log table

  • Print to Consoleread 2026-09-27

    Print a text to the console

  • Expose a macro's value as a value the Log operator can record

  • Step Custom Progressread 2026-09-27

    Advance the progress bar of a custom operator by one step

  • Total Custom Progressread 2026-09-27

    Set the total number of steps for a custom operator's progress bar

subtoken (1)
  • subtoken_cache_statsread 2026-09-27

    Return the vector cache's hits, misses, encoded, uncached, entries and capacity counts

Tableau Prep (2)
  • Descriptionsread 2026-09-27

    Add a description to a flow step or to a change entry

  • Group stepsread 2026-09-27

    Collapse a run of steps into a named group on the canvas

Talend (9)
  • tAssertCatcherread 2026-09-27

    Collect assertion and Job execution status records as a flow

  • tChronometerStartread 2026-09-27

    Start timing one or more subJobs

  • tChronometerStopread 2026-09-27

    Stop the timer and report the time one or more subJobs took

  • tFlowMeterread 2026-09-27

    Count the rows passing through a flow, for tFlowMeterCatcher to log

  • tFlowMeterCatcherread 2026-09-27

    Collect the row counts measured by tFlowMeter components as a flow

  • tJobLogread 2026-09-27

    Write a log message for the developer or for the user of the task

  • tLogCatcherread 2026-09-27

    Collect log records raised by Java exceptions, tDie or tWarn as a flow

  • tLogRowread 2026-09-27

    Print rows to the run console

  • tStatCatcherread 2026-09-27

    Collect Job and component processing statistics as a flow

Trifacta (2)
  • Commentread 2026-09-27

    Add a comment to the recipe

  • Comment Transformread 2026-09-27

    Insert a non-functional comment as a recipe step

VisiData (12)
  • cmdlog-allread 2026-09-23

    Open global CommandLog of the commands executed this session

  • cmdlog-sheetread 2026-09-23

    Open current sheet's CommandLog, including commands of its parent sheets

  • cmdlog-sheet-onlyread 2026-09-23

    Open CommandLog of current sheet without its parents' commands

  • open-input-historyread 2026-09-23

    Open sheet of previous inputs

  • open-statusesread 2026-09-23

    Open status message history

  • save-cmdlogread 2026-09-23

    Save the CommandLog to a .vdj file for later replay

  • vd (loader)read 2026-09-23

    Load command log in TSV form, for replay

  • vd (saver)read 2026-09-23

    Save as command log as TSV

  • vdj (loader)read 2026-09-23

    Load command log in JSON Lines form, for replay

  • vdj (saver)read 2026-09-23

    Save as command log as JSON Lines with a vd -p shebang

  • vdx (loader)read 2026-09-23

    Load command log in one-line-per-command text form, for replay

  • vdx (saver)read 2026-09-23

    Save as command log as one-line-per-command text

Yggdrasil Decision Forests (4)
  • strictread 2026-09-27

    Turn on strict mode, which displays more warnings

  • verboseread 2026-09-27

    Set how much training and loading detail is logged

  • ydf.help.loading_dataread 2026-09-27

    Print the list of dataset formats YDF reads

  • ydf.util.LogBookread 2026-09-27

    Record experiment keys and results under a directory and read them back as a DataFrame

Background

In computing, logging is keeping a log of events that occur in a system, such as problems, errors or broad information on current operations.[1] Log messages can be used to monitor and understand how a system operates, to debug problems, or during an audit.[1] In the simplest case, log messages are written to a file, called a log file.[1]

When to use it

Most database systems keep a transaction log, which records changes to the stored data so the database can recover from crashes, and which is not mainly intended as an audit trail.[1] A pipeline that needs a record of what each run did should write that record itself, rather than count on the database's own transaction log to tell the story later. (Meridian's view)

In Apache Hop, the Write to log action writes a chosen string to the Hop logging system, and its typical uses are logging variable values and custom logging information.[2] In DuckDB, logs may contain sensitive information, including secrets and credentials.[3] Log the counts, parameters and decisions a later reader will need, and keep credentials out of every message. (Meridian's view)

In Apache NiFi, the UpdateCounter processor sets counters at chosen points in a flow, and is useful for debugging and basic counting.[4] A counter at each stage of a run is the cheapest way to see where rows went missing. (Meridian's view)

In OpenRefine, any activity that changes the data can be undone, and each project's change history is saved with the project's data.[5] In OpenRefine, if you move back to an earlier state and then perform a new operation, the later actions are erased and cannot be re-applied.[5] In OpenRefine, operations can be extracted from one project as JSON and applied to another, though not all of them: an edit to a single cell cannot be replicated.[5]

The method

Event logs record events that take place as a system runs, and can be used to understand the system's activity and to diagnose problems.[1] A widely used logging standard is Syslog, defined in Request for Comments (RFC) 5424.[1][6] The Syslog Protocol, published as RFC 5424 in March 2009, is used to convey event notification messages.[6] RFC 5424 gives each Syslog message one of eight severities, numbered from 0, Emergency (system is unusable), to 7, Debug (debug-level messages).[6]

DuckDB's logging levels control how verbose its logs are, running from FATAL, which logs only fatal errors, through ERROR, WARNING, INFO and DEBUG to TRACE.[3] In DuckDB, the default log level is INFO, which logs general information, warnings and errors.[3] Choosing a level is choosing what a run will be able to tell you afterwards: a quiet default for scheduled runs, and a verbose level only while you are chasing a problem. (Meridian's view)