Control / flow · All operations

Limit the rate of rows

A variant of Branch or loop

Pass rows downstream no faster than a set number per second or minute.

Status

In arcform

Not built yet

arcform does not do this yet; it would live in how a Protocol runs. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Throttleread 2026-09-27

    Limit the number of records passed per minute to downstream tools

Apache NiFi (1)
  • ControlRateread 2026-09-27

    Throttle the rate at which data passes to downstream processors

Duckle (1)
  • Throttleread 2026-09-25

    delay the stage to a rows-per-second target, then pass rows through

Background

In computer networks, rate limiting is used to control the rate of requests sent or received by a network interface controller.[1] Apache NiFi's ControlRate processor controls the rate at which data is transferred to the processors that follow it.[2] Alteryx's Throttle tool slows downstream processing by limiting the number of records passed through it each minute.[3]

When to use it

Alteryx's Throttle tool is for sending data to or retrieving it from the web where requests have a rate limit.[3] When a client makes too many requests within a given time frame, an HTTP server can respond with status code 429: Too Many Requests.[1] Set the limit below the rate the service publishes, and pair it with a retry that waits longer after each refusal, since other clients may share the same limit. (Meridian's view)

Alteryx's Throttle tool sleeps at 1 millisecond intervals, so it cannot process more than 1,000 records per second.[3] In Apache NiFi, setting a very small Time Duration, the period over which ControlRate measures the rate, makes its throttle less accurate.[2]

The method

The token bucket algorithm adds tokens to a fixed-capacity bucket at a fixed rate, and a packet conforms only if the bucket holds enough tokens for it.[4] A flow that conforms to a token bucket can average up to the rate at which tokens are added, with its burstiness set by the depth of the bucket.[4] Packets that do not conform to a token bucket may be dropped, queued until enough tokens accumulate, or sent but marked as non-conformant.[4]

Apache NiFi's ControlRate can measure the rate as bytes per time period, as a count of FlowFiles, NiFi's single pieces of data, or as the accumulated value of an attribute.[2][5] When a Grouping Attribute is set, NiFi's ControlRate keeps a separate throttle for each value of that attribute.[2]

Compared with the alternatives

Exponential backoff reduces the rate of a process in response to adverse events, such as an error response from a service or an explicit request to reduce the rate.[6]

Sources

  1. [1]Rate limiting - Wikipediasecondary source
  2. [2]ControlRate - Apache NiFidocumentation
  3. [3]Throttle Tooldocumentation
  4. [4]Token bucket - Wikipediasecondary source
  5. [5]Apache NiFi User Guidedocumentation
  6. [6]Exponential backoff - Wikipediasecondary source