Control / subflow · All operations
Call a reusable workflow
Call a sub-workflow, macro, mapping or reusable fragment.
Status
- In arcform
Not built yet
arcform does not do this yet; it would live in how a Protocol runs. Its place on the roadmap
- Shape of the result
- not a data operation
- Level
- not a step
Variants
- Reuse steps inside the flow
Embed a saved group of steps, a sub-pipeline or a named subquery in the flow so rows pass through it as if its steps were written in place.
- Declare a workflow's inputs and outputs
Mark the points where a reusable workflow receives rows, files or variables from its caller and hands results back.
- Replay a recorded recipe of steps
Apply a saved sequence of preparation steps or recorded commands to the current data.
- Define a reusable macro or function
Write a parameterised macro or function once and call it from many queries or flows.
Fill a template pipeline at run time
Inject field names and settings from rows into a template pipeline, then run it.
Other reuse features
Deployment capture, remote groups, saved schemas and saved expressions that support reuse without calling a fragment.
Pass rows and file names between steps
Hand rows or a list of file names from one step of a workflow to a later one through the workflow result.
Run a workflow once per row
Call a child pipeline or workflow for each input row or group of rows, passing the row as parameters and collecting the results.
Run another workflow
Start a separate pipeline, job or process from this one, passing it parameters or a table.
What other tools call it
Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).
Alteryx Designer (4)
Macro Inputread 2026-09-27Declare an input anchor and its template data for a macro
Macro Input In-DBread 2026-09-27Declare an in-DB input connection on a macro, with placeholder values
Macro Outputread 2026-09-27Declare an output anchor for a macro
Macro Output In-DBread 2026-09-27Declare an in-DB output connection on a macro
Apache DataFusion (3)
Common Table Expressions (CTE)read 2026-09-27Name a subquery in a WITH clause and reference it in the query
Subqueriesread 2026-09-27Use a query inside SELECT, FROM, WHERE or HAVING
WITH RECURSIVEread 2026-09-27Recursive common table expression
Apache Hop (17)
Add filenames to resultread 2026-09-27Add files or folders to the result file list passed between actions
Copy rows to resultread 2026-09-27Hand rows in memory to the next pipeline or action of the calling workflow
Delete filenames from resultread 2026-09-27Remove file names from the result file list passed between actions
Files from resultread 2026-09-27Read the file names earlier actions or transforms added to the workflow result
Files to resultread 2026-09-27Add file names from the rows to the workflow result for later actions
Get Rows from Resultread 2026-09-27Read rows that an earlier pipeline in the workflow copied to the result
Get records from streamread 2026-09-27Deprecated: read rows passed from an earlier pipeline in the workflow
Injectorread 2026-09-27Entry point for rows injected through the Java API or a streaming consumer
Mapping inputread 2026-09-27Declare the input fields a mapping sub-pipeline receives
Mapping outputread 2026-09-27Declare the output fields a mapping sub-pipeline returns
Metadata Injectionread 2026-09-27Fill a template pipeline with metadata from rows at run time, then run it
Multi Mappingread 2026-09-27Run a reusable child pipeline with any number of input, output, info and target streams
Pipelineread 2026-09-27Run a pipeline from the workflow, passing parameters and result rows
Pipeline Executorread 2026-09-27Run another pipeline per row or group of rows, passing parameters and rows
Simple Mappingread 2026-09-27Run a reusable sub-pipeline with one input and one output inside the pipeline
Workflowread 2026-09-27Run another workflow from the current one
Workflow Executorread 2026-09-27Run a workflow per row or group of rows from inside a pipeline
Apache NiFi (4)
Input Portread 2026-09-27Entry point that transfers data into a Process Group
Output Portread 2026-09-27Exit point that transfers data out of a Process Group
Process Groupread 2026-09-27Group of components handled as one unit of the dataflow
Remote Process Groupread 2026-09-27Transfer data to or from another NiFi instance
AWS Glue (2)
Custom visual transformsread 2026-09-27User-built transforms, uploaded to Amazon S3, that then appear as nodes in the visual editor
Data preparation reciperead 2026-09-27Apply a recipe of data preparation steps authored in a grid or imported from AWS Glue DataBrew
Azure Data Factory (5)
Data Flowread 2026-09-27Run a mapping data flow on a Spark cluster that the service manages
Execute Pipelineread 2026-09-27Invoke another pipeline from a pipeline
Flowletread 2026-09-27Include a reusable block of transformation logic
Power Query activityread 2026-09-27Run a Power Query mash-up at scale on the data flow Spark environment
User defined functionsread 2026-09-27Reusable custom expression functions, kept in libraries and shared between data flows
Bruin (1)
Macros (user-defined)read 2026-09-27User Jinja macros in a macros/ directory, loaded automatically and called from SQL assets
dbt-utils (1)
ephemeralread 2026-09-27Build nothing; interpolate the model into dependent models as a CTE
DuckDB (1)
CREATE MACROread 2026-09-27Define a reusable scalar or table macro in SQL
Duckle (3)
Run Jobread 2026-09-25call a child pipeline with context variables
Run Pipelineread 2026-09-25run another pipeline file as a side effect, then pass rows through
Trigger Pipelineread 2026-09-25alias of Run Pipeline
EasyMorph (3)
Call another module/projectread 2026-09-27Run another module or project once, passing it the current table
Call with another tableread 2026-09-27Run another module or project, passing it another table
Inputread 2026-09-27Receive the dataset passed in by the module or project that called this one
KNIME Analytics Platform (16)
Call Workflow (Row Based)read 2026-09-27Call a local or remote workflow once per row, passing JSON and appending the results
Call Workflow (Table Based)read 2026-09-27Call a local or Hub workflow, passing tables and variables to its container nodes
Call Workflow Serviceread 2026-09-27Execute another workflow through its Workflow Input and Output ports and use the results
Container Input (Credentials)read 2026-09-27Receive credentials from an external caller of the workflow
Container Input (File)read 2026-09-27Receive a file from an external caller and expose its path as a flow variable
Container Input (JSON)read 2026-09-27Accept a JSON value from an external caller of the workflow, with a default from the dialog
Container Input (Row)read 2026-09-27Receive one row from an external caller of the workflow
Container Input (Table)read 2026-09-27Receive a table from an external caller of the workflow
Container Input (Variable)read 2026-09-27Receive flow variables from an external caller of the workflow
Container Output (File)read 2026-09-27Expose a file of the workflow to an external caller
Container Output (JSON)read 2026-09-27Expose a JSON column's content to an external caller of the workflow
Container Output (Row)read 2026-09-27Expose the first row of a table to an external caller
Container Output (Table)read 2026-09-27Expose a table to an external caller
KNIME Integrated Deployment (extension)read 2026-09-27Extension outside the base install: 8 current nodes: Capture Workflow End; Capture Workflow Start; Deploy Workflow to Server, …
Workflow Inputread 2026-09-27Receive a table or port object from a workflow that calls this one as a service
Workflow Outputread 2026-09-27Send a table or port object back to the workflow that called this one as a service
OpenRefine (3)
Apply…read 2026-09-27Paste operation JSON to run those operations on the current project
Extract…read 2026-09-27Pick operations from the history and get them encoded as JSON
Import schemaread 2026-09-27Load a saved schema from a JSON file or pasted JSON and bind it to columns of the same names
Pentaho Data Integration (16)
Copy rows to resultread 2026-09-27Pass rows to the executing job for its next entry to use
ETL metadata injectionread 2026-09-27Inject metadata into a template transformation before it runs
Get files from resultread 2026-09-27Read the file names used or produced by an earlier job entry
Get records from streamread 2026-09-27Return records produced earlier by another transformation in the job
Get rows from resultread 2026-09-27Read rows passed on by an earlier entry of the job
Injectorread 2026-09-27Entry point for rows injected into the transformation through the Java API
Job Executorread 2026-09-27Run a PDI job from the transformation, passing parameters and rows
MapReduce Inputread 2026-09-27Receive key-value pairs from Hadoop MapReduce into a mapper or reducer transformation
MapReduce Outputread 2026-09-27Return key-value pairs from a mapper or reducer transformation to Hadoop MapReduce
Mappingread 2026-09-27Run a sub-transformation whose interface is set by mapping input and output steps
Mapping Input Specificationread 2026-09-27Declare the input interface of a mapping sub-transformation
Mapping Output Specificationread 2026-09-27Declare the output interface of a mapping sub-transformation
Set files in resultread 2026-09-27Add file names to the transformation result for later job entries
Simple Mapping (sub-transformation)read 2026-09-27Wrap a reusable sequence of steps as a sub-transformation
Single Threaderread 2026-09-27Run a sequence of steps in a single thread
Transformation Executorread 2026-09-27Run a PDI transformation from the transformation, setting parameters and passing rows
Polars (3)
DataFrame.piperead 2026-09-27Pass the frame through a user function so custom steps chain like methods
Expr miscellaneous (group)read 2026-09-273 methods: deserialize, from_json, set_sorted; load a saved expression, or flag its output as sorted
LazyFrame.pipe_with_schemaread 2026-09-27Pass the lazy frame and its resolved schema to a user function at planning time
Power Query M (1)
Function valuesread 2026-09-276 functions: Function.Invoke, Function.InvokeAfter, Function.From, Function.ScalarVector, Function.IsDataSource
RapidMiner Studio (8)
Custom Subprocessread 2026-09-27Define a custom operator that exposes subprocesses to its user
Execute Processread 2026-09-27Run another stored process as a step of this one
Execute Process from Custom Extensionread 2026-09-27Run a process that is packaged inside a custom extension
Metadata Helperread 2026-09-27Define the output metadata of a custom operator via a subprocess, leaving data unchanged
Subprocessread 2026-09-27Group a chain of operators into a nested subprocess
Subprocess (Caching)read 2026-09-27Subprocess whose results are cached and reused on later runs
Subprocess (In Database)read 2026-09-27Group a chain of in-database operators into a nested subprocess
Work on Subsetread 2026-09-27Run a nested subprocess on a chosen subset of columns and merge the result back
SQLMesh (3)
@WITHread 2026-09-27Include a common table expression WITH clause only when the argument is true
EMBEDDEDread 2026-09-27Build nothing; inject the query as a subquery into each downstream model
Jinja macrosread 2026-09-27Jinja templating and user macro functions as the alternative macro system
Tableau Prep (1)
Insert Flowread 2026-09-27Insert steps saved from another flow as a reusable fragment
Talend (6)
tDataprepRunread 2026-09-27Apply a preparation built in Talend Data Preparation to the rows
tHMapread 2026-09-27Run a Talend Data Mapper map that transforms between source and destination structures
tHMapFileread 2026-09-27Run a Talend Data Mapper map over files as a Spark batch execution
tHMapInputread 2026-09-27Run a Talend Data Mapper map as a Spark batch execution and pass the output rows downstream
tHMapRecordread 2026-09-27Run a Talend Data Mapper map over records as a Spark streaming execution
tRunJobread 2026-09-27Run another Job in a given context
VisiData (5)
exec-<macroname>read 2026-09-23Run a saved macro; one command is registered per macro
macro-recordread 2026-09-23Start or stop recording commands into a macro bound to a keystroke or longname
macro-sheetread 2026-09-23Open index of saved macros; each opens as an editable command log
replay-allread 2026-09-23Replay the commands in the CommandLog sheet
replay-rowread 2026-09-23Replay the command in the current CommandLog row
Background
In programming, a function or subroutine is a callable unit of logic with a well-formed interface that can be invoked multiple times.[1] In Apache Hop, the Workflow action executes a previously defined workflow as a child workflow of the current one.[2] The Execute Pipeline activity allows an Azure Data Factory (ADF) pipeline to invoke another pipeline.[3]
When to use it
Apache Hop presents the Workflow action as a way to perform functional decomposition, breaking workflows out into more manageable units.[2] Apache Hop's guidance is that a data warehouse load is better built as smaller workflows composed together than as one workflow of 500 actions.[2] Reach for a called workflow when the same steps recur across pipelines, or when one pipeline has grown too long to read in one sitting. (Meridian's view)
In ADF, the Execute Pipeline activity passes parameters to the invoked pipeline as a JavaScript Object Notation (JSON) object that maps parameter names to argument values.[3] ADF's Execute Pipeline activity passes an array parameter to the child pipeline as a string.[3] Check the type a child receives, not only the value, whenever a parameter crosses from one pipeline to another. (Meridian's view)
In ADF, the waitOnCompletion property of Execute Pipeline defines whether the activity waits for the invoked pipeline to finish, and it defaults to true.[3] In Apache Hop, the Workflow action's option to wait for the child workflow blocks the parent until the child has finished, so the action returns the child's result.[2] Leave waiting on unless nothing later in the parent depends on the child's outcome. (Meridian's view)
The method
Maurice Wilkes, David Wheeler and Stanley Gill are generally credited with the formal invention of the subroutine, which they termed a closed sub-routine.[1] A closed sub-routine was contrasted with an open subroutine, or macro.[1] The idea of a callable unit was first conceived by John Mauchly and Kathleen Antonelli and recorded in a January 1947 Harvard symposium.[1]
In Apache Hop, the Workflow action's Parameters tab specifies which parameters are passed to the sub-workflow.[2] In Apache Hop, a sub-workflow's parameter value can be typed by hand, taken from another parameter, or combine text with parameter values.[2] In Apache Hop, variables are inherited by a sub-workflow whether or not the parent's values are passed to matching parameters.[2]
Apache Hop's Workflow action can execute the child workflow once for every result row returned by the previous action, passing one row to each execution.[2] In Apache Hop, the Copy Rows To Result transform transfers rows in memory to the next pipeline or workflow action in a workflow through an internal result row set.[4] In Apache Hop, Copy Rows To Result is mainly available for historical reasons, and newer ways to create loops exist.[4]
In Apache Hop, metadata injection provides a pipeline's required metadata at run time, which can reduce the need for slightly different versions of the same pipeline.[5] Apache Hop's Metadata Injection transform populates properties left unconfigured in a template pipeline, such as the fields of a Table input transform, at run time.[5]
Compared with the alternatives
Apache Hop separates its calls by caller: the Pipeline action executes a pipeline from a workflow, and the Workflow Executor transform executes a workflow from a pipeline.[2] In Apache Hop, the Pipeline Executor transform executes a sub-pipeline from a pipeline.[2] Pick the call that matches what the caller is and what it hands over: a whole run's parameters, or rows. (Meridian's view)
Sources
- [1]Function (computer programming)secondary source
- [2]Workflow | Apache Hopdocumentation
- [3]Execute Pipeline Activity - Azure Data Factory & Azure Synapse | Microsoft Learndocumentation
- [4]Copy rows to result | Apache Hopdocumentation
- [5]Metadata Injection | Apache Hopdocumentation
Used in
One Protocol uses this operation:
- EDGAR ↔ GLEIF — Company-to-LEI Crosswalk, steps
load,package