Shape / columns · All operations
Split a URL into its parts
A variant of Parse nested values
Extract the host, path, query parameters or other parts of a URL, or encode and decode URL text.
Status
- In arcform
Runs as a SQL step
arcform carries it as a
sql:step today.
The DuckDB step
Split a URL into its parts, read a query parameter, and encode and decode URL text
CREATE TEMP TABLE urls AS SELECT * FROM (VALUES
(1, 'https://shop.example.com:8080/a/b?q=red%20shoes&page=2#top'),
(2, 'http://example.org'),
(3, NULL)
) t(id, url);
CREATE TEMP TABLE result AS
WITH parts AS (
SELECT id, regexp_extract(url, '^(?:([a-z][a-z0-9+.-]*):)?(?://([^/?#:]*)(?::(\d+))?)?([^?#]*)(?:\?([^#]*))?(?:#(.*))?$',
['scheme', 'host', 'port', 'path', 'query', 'fragment']) AS p FROM urls)
SELECT id, p.*, url_decode(nullif(regexp_extract(p.query, '(?:^|&)q=([^&]*)', 1), '')) AS q_param,
url_encode('red shoes & more') AS encoded
FROM parts;
SELECT * FROM result ORDER BY id;
SELECT CASE WHEN ((SELECT [scheme, host, port, path, query, fragment, q_param] FROM result WHERE id = 1) = ['https', 'shop.example.com', '8080', '/a/b', 'q=red%20shoes&page=2', 'top', 'red shoes'] AND (SELECT [scheme, host, path, query] FROM result WHERE id = 2) = ['http', 'example.org', '', ''] AND (SELECT q_param FROM result WHERE id = 2) IS NULL AND (SELECT encoded FROM result WHERE id = 1) = 'red%20shoes%20%26%20more' AND (SELECT count(*) FROM result WHERE id = 3 AND host IS NULL) = 1) THEN 'as expected' ELSE error('the URLs were not split into scheme, host, port, path, query and fragment, or the q parameter was not decoded, or text was not encoded') END AS verdict;- The statement builds its own data.
- It needs nothing installed beyond DuckDB itself.
- The statement checks its own result and raises an error when it is wrong: “the URLs were not split into scheme, host, port, path, query and fragment, or the q parameter was not decoded, or text was not encoded”.
- It passed on DuckDB
v2.0.0 (Cyanoptera) d54cbbe3df.
What other tools call it
Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).
Bruin (3)
bruin.get_url_host(field)read 2026-09-27Emit an expression extracting the host from a URL string
bruin.get_url_parameter(field, parameter)read 2026-09-27Emit an expression extracting one query parameter value from a URL string
bruin.get_url_path(field)read 2026-09-27Emit an expression extracting the path, without query parameters, from a URL string
dbt-utils (3)
get_url_hostread 2026-09-27Extract the hostname from a URL column
get_url_parameterread 2026-09-27Extract a named query parameter from a URL column
get_url_pathread 2026-09-27Extract the page path from a URL column
Duckle (1)
URL Parseread 2026-09-25extract scheme, host, port, path, query or fragment from a URL by regex
EasyMorph (1)
Parse web form submissionread 2026-09-27Decode x-www-form-urlencoded form submissions into fields
Ibis (8)
ExtractAuthorityread 2026-09-27Extract the authority from a URL
ExtractFileread 2026-09-27Extract the file part from a URL
ExtractFragmentread 2026-09-27Extract the fragment from a URL
ExtractHostread 2026-09-27Extract the host from a URL
ExtractPathread 2026-09-27Extract the path from a URL
ExtractProtocolread 2026-09-27Extract the protocol from a URL
ExtractQueryread 2026-09-27Extract the query string, or one named parameter of it, from a URL
ExtractUserInforead 2026-09-27Extract the user info from a URL
OpenRefine (1)
URI parsingread 2026-09-271 function: parseUri
Power Query M (1)
Uri functionsread 2026-09-274 functions: Uri.Parts, Uri.BuildQueryString, Uri.Combine, Uri.EscapeDataString
RapidMiner Studio (2)
Decode URLsread 2026-09-27URL-decode the values of selected columns
Encode URLsread 2026-09-27URL-encode the values of selected columns
Trifacta (1)
Extract query stringsread 2026-09-27Extract fields from a URL query string
Background
Splitting a Uniform Resource Locator (URL) breaks it into its components: scheme, authority, path, query and fragment.[1] Percent-encoding writes a data octet as % followed by two hexadecimal digits, so %20 stands for the space character.[1]
When to use it
The URL Standard's application/x-www-form-urlencoded parser splits its input on & and takes each name up to the first =.[2] The application/x-www-form-urlencoded parser replaces each + in a name or value with a space before percent-decoding them.[2] Decide whether a + in a query value means a space before decoding it, and decode the whole column the same way. (Meridian's view)
The method
Unreserved characters need no percent-encoding: upper and lower case letters, decimal digits, hyphen, period, underscore and tilde.[1] DuckDB's url_encode and url_decode convert text to and from percent-encoding, so url_encode turns a space into %20 and / into %2F.[3]
The URL Standard aims to align Request for Comments (RFC) 3986 and RFC 3987 with contemporary implementations and to obsolete those RFCs in the process.[2]
Sources
- [1]RFC 3986: Uniform Resource Identifier (URI): Generic Syntaxstandard
- [2]URL Standardstandard
- [3]Text Functions – DuckDBdocumentation