Shape / rows · All operations

Keep rows that contain text

A variant of Filter rows

Keep or drop the rows whose cells contain a string or match a pattern, in one column or any.

Status

In arcform

Runs as a SQL step

arcform carries it as a sql: step today.

The DuckDB step

Keep or drop rows that contain a string or match a pattern, in one column or any

It reads the orders table the proofs share. Create it first:

SQL
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);

Then run the statement:

SQL
-- one column, a string (case-insensitive)
CREATE TEMP TABLE has_string AS SELECT order_id FROM orders WHERE contains(lower(customer), 'alice');
-- one column, a pattern
CREATE TEMP TABLE has_pattern AS SELECT order_id FROM orders WHERE regexp_matches(customer, '^(Dev|Eve) ');
-- the rows that do not contain the string
CREATE TEMP TABLE lacks_string AS SELECT order_id FROM orders WHERE NOT contains(lower(customer), 'alice');
-- any text column
CREATE TEMP TABLE in_any_column AS
SELECT order_id FROM orders
WHERE customer ILIKE '%phone%' OR region ILIKE '%phone%' OR product ILIKE '%phone%' OR tags ILIKE '%phone%' OR meta ILIKE '%phone%';
-- the same word in the customer column alone finds nothing, so any column differs from one
CREATE TEMP TABLE in_customer_only AS SELECT order_id FROM orders WHERE customer ILIKE '%phone%';
SELECT list(order_id ORDER BY order_id) AS has_string FROM has_string;
SELECT list(order_id ORDER BY order_id) AS in_any_column FROM in_any_column;
SELECT CASE WHEN ((SELECT list(order_id ORDER BY order_id) FROM has_string) = [1, 2, 10] AND (SELECT list(order_id ORDER BY order_id) FROM has_pattern) = [6, 8, 9] AND (SELECT count(*) FROM lacks_string) = 7 AND (SELECT list(order_id ORDER BY order_id) FROM in_any_column) = [5, 10] AND (SELECT count(*) FROM in_customer_only) = 0) THEN 'as expected' ELSE error('the string, the pattern, the exclusion or the any-column match did not return the expected orders') END AS verdict;
  • It needs nothing installed beyond DuckDB itself.
  • The statement checks its own result and raises an error when it is wrong: “the string, the pattern, the exclusion or the any-column match did not return the expected orders”.
  • It passed on DuckDB v2.0.0 (Cyanoptera) d54cbbe3df.

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Apache NiFi (1)
  • Filter Functionsread 2026-09-27

    8 functions: contains, containsRegex, matchesRegex, startsWith, endsWith, …

EasyMorph (1)
OpenRefine (1)
  • Text filterread 2026-09-27

    Keep rows whose cell contains a typed string or matches a regular expression; can be inverted

Power Query M (1)
  • Table.FindTextread 2026-09-27

    Keep rows where any cell contains the given text

Tableau Prep (1)
  • Wildcard Matchread 2026-09-27

    Keep or exclude string values matching a pattern

Trifacta (3)
  • Filter containsread 2026-09-27

    Keep or delete rows whose value contains a literal or pattern

  • Filter ends withread 2026-09-27

    Keep or delete rows whose value ends with a literal or pattern

  • Filter starts withread 2026-09-27

    Keep or delete rows whose value starts with a literal or pattern

VisiData (2)
  • dup-searchread 2026-09-23

    (experimental) Open duplicate sheet filtered live to rows matching a regex in current column

  • dup-search-colsread 2026-09-23

    (experimental) Open duplicate sheet filtered live to rows matching a regex in any column

Background

In PostgreSQL and DuckDB, LIKE pattern matching always covers the entire string, so a pattern for text anywhere must start and end with a percent sign.[1][2] In PostgreSQL and DuckDB, an underscore in a LIKE pattern matches any single character, and a percent sign matches any sequence of zero or more characters.[1][2] In DuckDB, contains(string, search_string) returns true if search_string is found within string.[3]

When to use it

In PostgreSQL, the key word ILIKE makes the match case-insensitive under the active locale, and is a PostgreSQL extension rather than part of the SQL standard.[1] In DuckDB, ILIKE makes a LIKE match case-insensitive under the active locale.[2] In PostgreSQL, a literal underscore or percent sign in a LIKE pattern must be preceded by the escape character, which is the backslash by default.[1] When the search text comes from a person or a file, escape any percent sign or underscore in it before building a LIKE pattern, or use a plain containment function, so that a code such as AB_12 matches only itself. (Meridian's view)

In PostgreSQL, regular expressions can be contrived that take arbitrary amounts of time and memory to process, and LIKE searches, being much simpler, are safer with possibly hostile patterns.[1] Searching every text column for a word can find it in columns you did not mean, such as notes or tags, so name the columns when you know which ones hold the text. (Meridian's view)

The method

In DuckDB, regexp_matches returns true if the string contains the pattern anywhere, while regexp_full_match returns true only if the entire string matches.[4] DuckDB uses the RE2 library as its regular expression engine.[4] In DuckDB, regexp_matches is optimised to the LIKE operator when possible, and the c option for case-sensitive matching should be passed where applicable for the best performance.[4] In DuckDB, the RE2 library by default does not match the . character to a newline.[4] In PostgreSQL, a LIKE pattern with no percent signs or underscores represents only the string itself, so LIKE then acts like the equals operator.[1]

Compared with the alternatives

In DuckDB, IN can be used with a string on its right as a shorthand for the contains function, as in 'Hello' IN 'Hello World'.[5] In DuckDB, the contains function does not support collations.[3] In PostgreSQL, SIMILAR TO provides many of the same capabilities as POSIX-style regular expressions, and searches using it have the same security hazards.[1]