Shape / rows · All operations
Keep rows that contain text
A variant of Filter rows
Keep or drop the rows whose cells contain a string or match a pattern, in one column or any.
Status
- In arcform
Runs as a SQL step
arcform carries it as a
sql:step today.
The DuckDB step
Keep or drop rows that contain a string or match a pattern, in one column or any
It reads the orders table the proofs share. Create it first:
CREATE TABLE orders AS SELECT * FROM (VALUES
(1, 'Alice Ng', 'north', DATE '2026-01-03', 120.00, 'widget', 'a|b', '{"channel":"web","items":[1,2]}'),
(2, 'alice ng', 'north', DATE '2026-01-05', 80.50, 'gadget', 'b', '{"channel":"shop","items":[3]}'),
(3, 'Bob Okafor', 'south', DATE '2026-01-05', 300.00, 'widget', 'a|c', '{"channel":"web","items":[4,5,6]}'),
(4, 'Carla Díaz', 'east', DATE '2026-01-09', NULL, 'gizmo', NULL, '{"channel":"web","items":[]}'),
(5, 'Bob Okafor', 'south', DATE '2026-01-12', 45.25, 'gadget', 'c', '{"channel":"phone","items":[7]}'),
(6, 'Dev Patel', 'west', DATE '2026-01-12', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'),
(7, 'Carla Diaz', 'east', DATE '2026-01-20', 99.99, 'gizmo', 'b|c', '{"channel":"shop","items":[10]}'),
(8, 'Dev Patel', 'west', DATE '2026-02-02', 510.00, 'widget', 'a', '{"channel":"web","items":[8,9]}'),
(9, 'Eve Larsen', NULL, DATE '2026-02-03', 15.00, 'gadget', NULL, '{"channel":"web","items":[11]}'),
(10, 'Alice Ng', 'north', DATE '2026-02-10', 220.00, 'gizmo', 'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);Then run the statement:
-- one column, a string (case-insensitive)
CREATE TEMP TABLE has_string AS SELECT order_id FROM orders WHERE contains(lower(customer), 'alice');
-- one column, a pattern
CREATE TEMP TABLE has_pattern AS SELECT order_id FROM orders WHERE regexp_matches(customer, '^(Dev|Eve) ');
-- the rows that do not contain the string
CREATE TEMP TABLE lacks_string AS SELECT order_id FROM orders WHERE NOT contains(lower(customer), 'alice');
-- any text column
CREATE TEMP TABLE in_any_column AS
SELECT order_id FROM orders
WHERE customer ILIKE '%phone%' OR region ILIKE '%phone%' OR product ILIKE '%phone%' OR tags ILIKE '%phone%' OR meta ILIKE '%phone%';
-- the same word in the customer column alone finds nothing, so any column differs from one
CREATE TEMP TABLE in_customer_only AS SELECT order_id FROM orders WHERE customer ILIKE '%phone%';
SELECT list(order_id ORDER BY order_id) AS has_string FROM has_string;
SELECT list(order_id ORDER BY order_id) AS in_any_column FROM in_any_column;
SELECT CASE WHEN ((SELECT list(order_id ORDER BY order_id) FROM has_string) = [1, 2, 10] AND (SELECT list(order_id ORDER BY order_id) FROM has_pattern) = [6, 8, 9] AND (SELECT count(*) FROM lacks_string) = 7 AND (SELECT list(order_id ORDER BY order_id) FROM in_any_column) = [5, 10] AND (SELECT count(*) FROM in_customer_only) = 0) THEN 'as expected' ELSE error('the string, the pattern, the exclusion or the any-column match did not return the expected orders') END AS verdict;- It needs nothing installed beyond DuckDB itself.
- The statement checks its own result and raises an error when it is wrong: “the string, the pattern, the exclusion or the any-column match did not return the expected orders”.
- It passed on DuckDB
v2.0.0 (Cyanoptera) d54cbbe3df.
What other tools call it
Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).
Apache NiFi (1)
Filter Functionsread 2026-09-278 functions: contains, containsRegex, matchesRegex, startsWith, endsWith, …
EasyMorph (1)
Filter by searchread 2026-09-27Keep or remove rows that contain a search string
OpenRefine (1)
Text filterread 2026-09-27Keep rows whose cell contains a typed string or matches a regular expression; can be inverted
Power Query M (1)
Table.FindTextread 2026-09-27Keep rows where any cell contains the given text
Tableau Prep (1)
Wildcard Matchread 2026-09-27Keep or exclude string values matching a pattern
Trifacta (3)
Filter containsread 2026-09-27Keep or delete rows whose value contains a literal or pattern
Filter ends withread 2026-09-27Keep or delete rows whose value ends with a literal or pattern
Filter starts withread 2026-09-27Keep or delete rows whose value starts with a literal or pattern
VisiData (2)
dup-searchread 2026-09-23(experimental) Open duplicate sheet filtered live to rows matching a regex in current column
dup-search-colsread 2026-09-23(experimental) Open duplicate sheet filtered live to rows matching a regex in any column
Background
In PostgreSQL and DuckDB, LIKE pattern matching always covers the entire string, so a pattern for text anywhere must start and end with a percent sign.[1][2] In PostgreSQL and DuckDB, an underscore in a LIKE pattern matches any single character, and a percent sign matches any sequence of zero or more characters.[1][2] In DuckDB, contains(string, search_string) returns true if search_string is found within string.[3]
When to use it
In PostgreSQL, the key word ILIKE makes the match case-insensitive under the active locale, and is a PostgreSQL extension rather than part of the SQL standard.[1] In DuckDB, ILIKE makes a LIKE match case-insensitive under the active locale.[2] In PostgreSQL, a literal underscore or percent sign in a LIKE pattern must be preceded by the escape character, which is the backslash by default.[1] When the search text comes from a person or a file, escape any percent sign or underscore in it before building a LIKE pattern, or use a plain containment function, so that a code such as AB_12 matches only itself. (Meridian's view)
In PostgreSQL, regular expressions can be contrived that take arbitrary amounts of time and memory to process, and LIKE searches, being much simpler, are safer with possibly hostile patterns.[1] Searching every text column for a word can find it in columns you did not mean, such as notes or tags, so name the columns when you know which ones hold the text. (Meridian's view)
The method
In DuckDB, regexp_matches returns true if the string contains the pattern anywhere, while regexp_full_match returns true only if the entire string matches.[4] DuckDB uses the RE2 library as its regular expression engine.[4] In DuckDB, regexp_matches is optimised to the LIKE operator when possible, and the c option for case-sensitive matching should be passed where applicable for the best performance.[4] In DuckDB, the RE2 library by default does not match the . character to a newline.[4] In PostgreSQL, a LIKE pattern with no percent signs or underscores represents only the string itself, so LIKE then acts like the equals operator.[1]
Compared with the alternatives
In DuckDB, IN can be used with a string on its right as a shorthand for the contains function, as in 'Hello' IN 'Hello World'.[5] In DuckDB, the contains function does not support collations.[3] In PostgreSQL, SIMILAR TO provides many of the same capabilities as POSIX-style regular expressions, and searches using it have the same security hazards.[1]
Sources
- [1]PostgreSQL: Documentation: 18: 9.7. Pattern Matchingdocumentation
- [2]Pattern Matching – DuckDBdocumentation
- [3]Text Functions – DuckDBdocumentation
- [4]Regular Expressions – DuckDBdocumentation
- [5]IN Operator – DuckDBdocumentation