Control / log · All operations

Read run logs and metrics as rows

A variant of Log and annotate a run

Return the run's log records, step timings, row counts or engine statistics as a table that later steps can query.

Status

In arcform

Not built yet

arcform does not do this yet; it would live in how a Protocol runs. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Apache DataFusion (3)
  • list_files_cacheread 2026-09-27

    Table function listing the contents of the listed-files cache

  • metadata_cacheread 2026-09-27

    Table function listing the contents of the file metadata cache

  • statistics_cacheread 2026-09-27

    Table function listing the contents of the file statistics cache

Apache Hop (7)
  • Execution informationread 2026-09-27

    Read or delete execution records from an execution information location

  • Neo4j Get Logging Inforead 2026-09-27

    Read execution logging information stored in a Neo4j logging graph

  • Output transform metricsread 2026-09-27

    Wait for named transforms to finish and emit their row counts and timings as rows

  • Pipeline Data Proberead 2026-09-27

    Receive rows sampled from other pipelines by a pipeline probe

  • Pipeline loggingread 2026-09-27

    Receive logging records about pipeline executions and their transforms

  • Server Statusread 2026-09-27

    Query the Hop servers named in a field and return their status

  • Workflow loggingread 2026-09-27

    Receive logging records about workflow executions and their actions

DuckDB (3)
  • duckdb_log_contextsread 2026-09-27

    duckdb_log_contexts table function

  • duckdb_logsread 2026-09-27

    duckdb_logs table function

  • duckdb_logs_parsedread 2026-09-27

    duckdb_logs_parsed table function

Duckle (1)
  • Run Eventsread 2026-09-25

    rows describing the stages that have already failed in this run

KNIME Analytics Platform (2)
  • Global Timer Inforead 2026-09-27

    Report node usage and timing collected over the life of the installation

  • Timer Inforead 2026-09-27

    Report execution timing for every node of the workflow

Pentaho Data Integration (1)
RapidMiner Studio (7)
  • Get Jobs (AI Hub)read 2026-09-27

    List the jobs executed on an AI Hub server as a table

  • Get Log (AI Hub)read 2026-09-27

    Fetch the execution log of a job on an AI Hub server

  • Get Log (RTSA)read 2026-09-27

    Fetch the contents of a log file from a Real-Time Scoring Agent

  • Get Logs (RTSA)read 2026-09-27

    List the log files of a Real-Time Scoring Agent

  • Get Metrics (AI Hub)read 2026-09-27

    Fetch the current health metrics of an AI Hub server

  • Log to Dataread 2026-09-27

    Convert a log table into a data table

  • Log to Weightsread 2026-09-27

    Build feature weights from column names recorded in a log table

subtoken (1)
  • subtoken_cache_statsread 2026-09-27

    Return the vector cache's hits, misses, encoded, uncached, entries and capacity counts

Talend (4)
  • tAssertCatcherread 2026-09-27

    Collect assertion and Job execution status records as a flow

  • tFlowMeterCatcherread 2026-09-27

    Collect the row counts measured by tFlowMeter components as a flow

  • tLogCatcherread 2026-09-27

    Collect log records raised by Java exceptions, tDie or tWarn as a flow

  • tStatCatcherread 2026-09-27

    Collect Job and component processing statistics as a flow

Background

In DuckDB, logs are stored in a view named duckdb_logs, which can be queried like any standard table.[1] In RapidMiner, the Log to Data operator turns what the Log operator recorded into an ExampleSet, a data table other operators of the process can use.[2] In Apache Hop, the Pipeline logging transform provides detailed logging information about a pipeline and its transforms.[3]

When to use it

In RapidMiner, the information the Log operator stores can be viewed in the Results View but is not directly accessible in the process.[2] Once a run's records are rows, a later step can test them like any other data: fail the run when a step rejected rows, or compare today's counts with yesterday's. (Meridian's view)

In Apache Hop, the Find previous successful execution operation of the Execution information transform answers when a workflow or pipeline last succeeded, for incremental extracts that filter on a timestamp or an increasing ID.[4] In Apache Hop, the Execution information transform always needs input rows in order to produce output.[4]

In Apache Hop, Output transform metrics waits until the transforms it watches have finished, then emits one row per watched copy of a transform with the requested counters.[5] In Apache Hop, Output transform metrics does not consume incoming rows, and connecting an upstream transform to it can deadlock the pipeline, because the watched transform cannot finish while its output buffer is full.[5] In Apache Hop, Output transform metrics is meant for a pipeline that itself needs metrics, while Pipeline logging or Execution information suit project-wide monitoring.[5]

In the DuckDB command-line client, the default log storage prints log messages to the console and cannot be queried, so duckdb_logs returns an error until logging is switched to a queryable storage such as memory or file.[1] In DuckDB, the duckdb_logs view targets the currently active log storage, so switching storage may change what it returns.[1]

The method

In Apache Hop, Pipeline logging can output one line per transform, with the pipeline's logging and metrics repeated on each line.[3] In Apache Hop, the fields Pipeline logging writes for each transform include its input, output, updated and rejected lines, its number of errors, and its start time, end time and total duration.[3]

In DuckDB, some log types, such as HTTP, have an associated message schema, and the duckdb_logs_parsed() macro parses their messages automatically.[1] In DuckDB, the parsed messages of the Metrics log type have two VARCHAR columns, metric and value.[1] In DuckDB's normalized logging, log entries are stored separately from their context information and reference it through context_ids, while denormalized logging appends the context to each entry.[1]

In RapidMiner, the Log operator can store almost any information in its log table, including the parameter values of operators, how often an operator was applied and execution times.[2]

Sources

  1. [1]Logging – DuckDBdocumentation
  2. [2]Log to Data - Rapidminer Documentationdocumentation
  3. [3]Pipeline logging | Apache Hopdocumentation
  4. [4]Execution information | Apache Hopdocumentation
  5. [5]Output transform metrics | Apache Hopdocumentation