Control / log ยท All operations
Log and annotate a run
Logging, audit, metrics, annotations, comments.
Status
- In arcform
A feature, not a step
It lives in how a Protocol runs. src/contract.rs in arcform at fc62d6f
- Shape of the result
- not a data operation
- Level
- not a step
Variants
- Annotate a pipeline
Attach comments, notes, descriptions or groupings to the steps of a pipeline without affecting what it computes.
- Explain or profile a query plan
Show the plan a query will run, or run it and report where the time went.
- Read run logs and metrics as rows
Return the run's log records, step timings, row counts or engine statistics as a table that later steps can query.
Track experiments and runs
Record experiment runs with their parameters and results in a tracking store and read them back.
Keep a replayable command history
Record every command or change of a session so it can be reviewed, undone or replayed.
Set how much is logged
Turn logging, profiling or verbose output on or off for a run.
Count, time and publish run metrics
Increment counters, record gauges and timers, and publish metrics or progress while a run proceeds.
Write a message to the log
Write a message, warning or the passing rows to the run log or console, and pass the data through unchanged.
What other tools call it
Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).
Alteryx Designer (5)
API Outputread 2026-09-27Print the stream to the Results window as quoted comma-delimited text for API callers
Commentread 2026-09-27Annotate the canvas with a note
Explorer Boxread 2026-09-27Show a web page, directory or file in a box on the canvas as supporting documentation
Messageread 2026-09-27Write a message, warning or error to the Results window from an expression
Tool Containerread 2026-09-27Group tools on the canvas; a container can be collapsed or disabled
Apache DataFusion (6)
EXPLAINread 2026-09-27Show the logical and physical plan of a statement: tree, indent, pgjson or graphviz
EXPLAIN ANALYZEread 2026-09-27Run the statement and show its plan with execution metrics
explainread 2026-09-27DataFrame method: Return the plan of the DataFrame so far
list_files_cacheread 2026-09-27Table function listing the contents of the listed-files cache
metadata_cacheread 2026-09-27Table function listing the contents of the file metadata cache
statistics_cacheread 2026-09-27Table function listing the contents of the file statistics cache
Apache Hop (9)
Execution informationread 2026-09-27Read or delete execution records from an execution information location
Neo4j Get Logging Inforead 2026-09-27Read execution logging information stored in a Neo4j logging graph
Output transform metricsread 2026-09-27Wait for named transforms to finish and emit their row counts and timings as rows
Pipeline Data Proberead 2026-09-27Receive rows sampled from other pipelines by a pipeline probe
Pipeline loggingread 2026-09-27Receive logging records about pipeline executions and their transforms
Server Statusread 2026-09-27Query the Hop servers named in a field and return their status
Workflow loggingread 2026-09-27Receive logging records about workflow executions and their actions
Write to Logread 2026-09-27Write a message to the log at a chosen level
Write to logread 2026-09-27Write chosen field values and a message to the log
Apache NiFi (10)
DebugFlowread 2026-09-27Testing aid: trigger chosen framework responses when a FlowFile or timer event arrives
Labelread 2026-09-27Documentation text placed on the canvas beside parts of a dataflow
LogAttributeread 2026-09-27Write the attributes of a FlowFile to the log at a chosen level
LogMessageread 2026-09-27Write a message to the log at a chosen level
LoggingRecordSinkread 2026-09-27Record sink: write records to the application log
MonitorActivityread 2026-09-27Signal when the flow has had no data for a set time, and again when activity resumes
PutCloudWatchMetricread 2026-09-27Publish a metric value or statistic set to Amazon CloudWatch
Reporting Tasks (family)read 2026-09-2710 components that publish metrics, status, bulletins and provenance events, or monitor disk and memory
UpdateCounterread 2026-09-27Increment a named counter, for debugging and basic counting in the flow
UpdateGaugeread 2026-09-27Record a value for a named gauge for each FlowFile processed
dbt-utils (3)
log_inforead 2026-09-27Log a timestamped message to the command line
pretty_log_formatread 2026-09-27Format a message with a timestamp prefix for command-line logging
pretty_timeread 2026-09-27Return the current timestamp as a formatted string at compile time
DuckDB (10)
Profilingread 2026-09-27Report where a query spent its time
disable_loggingread 2026-09-27disable_logging table function
disable_profilingread 2026-09-27disable_profiling table function
duckdb_log_contextsread 2026-09-27duckdb_log_contexts table function
duckdb_logsread 2026-09-27duckdb_logs table function
duckdb_logs_parsedread 2026-09-27duckdb_logs_parsed table function
duckdb_profiling_settingsread 2026-09-27duckdb_profiling_settings table function
enable_loggingread 2026-09-27enable_logging table function
enable_profilingread 2026-09-27enable_profiling table function
truncate_duckdb_logsread 2026-09-27truncate_duckdb_logs table function
Duckle (3)
Log Messageread 2026-09-25write an info log line with the upstream row count, then pass rows through
Run Eventsread 2026-09-25rows describing the stages that have already failed in this run
Warnread 2026-09-25write a warning log line, then pass rows through
EasyMorph (3)
Generate documentationread 2026-09-27Generate HTML documentation of an EasyMorph project
Statusread 2026-09-27Set a custom execution status message, optionally computed by an expression
To-doread 2026-09-27Leave a to-do note in the workflow
KNIME Analytics Platform (2)
Global Timer Inforead 2026-09-27Report node usage and timing collected over the life of the installation
Timer Inforead 2026-09-27Report execution timing for every node of the workflow
mlpack (2)
mlpack_silentread 2026-09-27Boolean setting that suppresses the short summaries the functions print, such as Misclassified
mlpack_verboseread 2026-09-27Boolean setting that prints the input matrices and fitted values while a function runs
OpenRefine (1)
History (Undo/Redo)read 2026-09-27Keep every data change as a numbered list saved with the project; click an entry to go to that state
Pentaho Data Integration (3)
Output steps metricsread 2026-09-27Return run metrics for one or several steps as rows
Send message to Syslogread 2026-09-27Send a message to a syslog server
Write to logread 2026-09-27Write row data to the log
Polars (3)
Expr manipulation/selection: utilities (group)read 2026-09-273 expressions: inspect, pipe, rechunk; print a value in passing, chain a user function, compact memory
Expr.meta (group)read 2026-09-2716 methods: output_name, root_names, is_column, tree_format, serialize and others; inspect the expression itself
LazyFrame query plan inspection (family)read 2026-09-274 methods: explain, show_graph, profile, inspect; print, draw or time the query plan and its nodes
RapidMiner Studio (26)
Annotateread 2026-09-27Add or change key-value annotations on an object
Annotations to Dataread 2026-09-27Extract an object's annotations into a table
Clear Logread 2026-09-27Clear a log table created by the Log operator
Create Experimentread 2026-09-27Create an experiment on an MLflow tracking server
Create Runread 2026-09-27Create a run inside an existing MLflow experiment
Data to Annotationsread 2026-09-27Set an object's annotations from the rows of a table
Delete Experimentread 2026-09-27Delete an experiment on an MLflow tracking server
Delete Runread 2026-09-27Delete an MLflow run
Extract Log Valueread 2026-09-27Expose a cell value of a table as a value the Log operator can record
Get Experimentsread 2026-09-27List the experiments on an MLflow tracking server
Get Jobs (AI Hub)read 2026-09-27List the jobs executed on an AI Hub server as a table
Get Log (AI Hub)read 2026-09-27Fetch the execution log of a job on an AI Hub server
Get Log (RTSA)read 2026-09-27Fetch the contents of a log file from a Real-Time Scoring Agent
Get Logs (RTSA)read 2026-09-27List the log files of a Real-Time Scoring Agent
Get Metrics (AI Hub)read 2026-09-27Fetch the current health metrics of an AI Hub server
Get Runread 2026-09-27Fetch one MLflow run by its run id
Get Runsread 2026-09-27List the runs of an MLflow experiment
Logread 2026-09-27Record operator values and parameters to a log table on each execution
Log Operator Runtimesread 2026-09-27Turn on logging of each operator's runtime for the current process
Log Runread 2026-09-27Log model evaluation results (metrics, parameters) into an existing MLflow run
Log to Dataread 2026-09-27Convert a log table into a data table
Log to Weightsread 2026-09-27Build feature weights from column names recorded in a log table
Print to Consoleread 2026-09-27Print a text to the console
Provide Macro as Log Valueread 2026-09-27Expose a macro's value as a value the Log operator can record
Step Custom Progressread 2026-09-27Advance the progress bar of a custom operator by one step
Total Custom Progressread 2026-09-27Set the total number of steps for a custom operator's progress bar
subtoken (1)
subtoken_cache_statsread 2026-09-27Return the vector cache's hits, misses, encoded, uncached, entries and capacity counts
Tableau Prep (2)
Descriptionsread 2026-09-27Add a description to a flow step or to a change entry
Group stepsread 2026-09-27Collapse a run of steps into a named group on the canvas
Talend (9)
tAssertCatcherread 2026-09-27Collect assertion and Job execution status records as a flow
tChronometerStartread 2026-09-27Start timing one or more subJobs
tChronometerStopread 2026-09-27Stop the timer and report the time one or more subJobs took
tFlowMeterread 2026-09-27Count the rows passing through a flow, for tFlowMeterCatcher to log
tFlowMeterCatcherread 2026-09-27Collect the row counts measured by tFlowMeter components as a flow
tJobLogread 2026-09-27Write a log message for the developer or for the user of the task
tLogCatcherread 2026-09-27Collect log records raised by Java exceptions, tDie or tWarn as a flow
tLogRowread 2026-09-27Print rows to the run console
tStatCatcherread 2026-09-27Collect Job and component processing statistics as a flow
Trifacta (2)
Commentread 2026-09-27Add a comment to the recipe
Comment Transformread 2026-09-27Insert a non-functional comment as a recipe step
VisiData (12)
cmdlog-allread 2026-09-23Open global CommandLog of the commands executed this session
cmdlog-sheetread 2026-09-23Open current sheet's CommandLog, including commands of its parent sheets
cmdlog-sheet-onlyread 2026-09-23Open CommandLog of current sheet without its parents' commands
open-input-historyread 2026-09-23Open sheet of previous inputs
open-statusesread 2026-09-23Open status message history
save-cmdlogread 2026-09-23Save the CommandLog to a .vdj file for later replay
vd (loader)read 2026-09-23Load command log in TSV form, for replay
vd (saver)read 2026-09-23Save as command log as TSV
vdj (loader)read 2026-09-23Load command log in JSON Lines form, for replay
vdj (saver)read 2026-09-23Save as command log as JSON Lines with a vd -p shebang
vdx (loader)read 2026-09-23Load command log in one-line-per-command text form, for replay
vdx (saver)read 2026-09-23Save as command log as one-line-per-command text
Yggdrasil Decision Forests (4)
strictread 2026-09-27Turn on strict mode, which displays more warnings
verboseread 2026-09-27Set how much training and loading detail is logged
ydf.help.loading_dataread 2026-09-27Print the list of dataset formats YDF reads
ydf.util.LogBookread 2026-09-27Record experiment keys and results under a directory and read them back as a DataFrame
Background
In computing, logging is keeping a log of events that occur in a system, such as problems, errors or broad information on current operations.[1] Log messages can be used to monitor and understand how a system operates, to debug problems, or during an audit.[1] In the simplest case, log messages are written to a file, called a log file.[1]
When to use it
Most database systems keep a transaction log, which records changes to the stored data so the database can recover from crashes, and which is not mainly intended as an audit trail.[1] A pipeline that needs a record of what each run did should write that record itself, rather than count on the database's own transaction log to tell the story later. (Meridian's view)
In Apache Hop, the Write to log action writes a chosen string to the Hop logging system, and its typical uses are logging variable values and custom logging information.[2] In DuckDB, logs may contain sensitive information, including secrets and credentials.[3] Log the counts, parameters and decisions a later reader will need, and keep credentials out of every message. (Meridian's view)
In Apache NiFi, the UpdateCounter processor sets counters at chosen points in a flow, and is useful for debugging and basic counting.[4] A counter at each stage of a run is the cheapest way to see where rows went missing. (Meridian's view)
In OpenRefine, any activity that changes the data can be undone, and each project's change history is saved with the project's data.[5] In OpenRefine, if you move back to an earlier state and then perform a new operation, the later actions are erased and cannot be re-applied.[5] In OpenRefine, operations can be extracted from one project as JSON and applied to another, though not all of them: an edit to a single cell cannot be replicated.[5]
The method
Event logs record events that take place as a system runs, and can be used to understand the system's activity and to diagnose problems.[1] A widely used logging standard is Syslog, defined in Request for Comments (RFC) 5424.[1][6] The Syslog Protocol, published as RFC 5424 in March 2009, is used to convey event notification messages.[6] RFC 5424 gives each Syslog message one of eight severities, numbered from 0, Emergency (system is unusable), to 7, Debug (debug-level messages).[6]
DuckDB's logging levels control how verbose its logs are, running from FATAL, which logs only fatal errors, through ERROR, WARNING, INFO and DEBUG to TRACE.[3] In DuckDB, the default log level is INFO, which logs general information, warnings and errors.[3] Choosing a level is choosing what a run will be able to tell you afterwards: a quiet default for scheduled runs, and a verbose level only while you are chasing a problem. (Meridian's view)
Sources
- [1]Logging (computing) - Wikipediasecondary source
- [2]Write to Log | Apache Hopdocumentation
- [3]Logging โ DuckDBdocumentation
- [4]UpdateCounter - Apache NiFidocumentation
- [5]Running OpenRefine | OpenRefinedocumentation
- [6]The Syslog Protocolstandard