Shape operations
Operations that clean, inspect and rearrange one table. Some work on its columns, such as changing a column's type, parsing dates, filling in missing values or adding a calculated column. Some work on its rows, such as filtering, sorting, sampling or removing duplicates. Some change the table as a whole, such as summarising by group or pivoting, and some only look at it, such as profiling the table or checking its quality.
51 operations, grouped by the shape of the result, each with its status in arcform today.
One row out per row in, with columns added or changed
Group values into binsRuns as a SQL step
Discretise, bucket, tile, histogram bins (2 variants with a page)
Convert binary and encoded textRuns as a SQL step
Base64, hex, blob and character-set conversion
Make a column from examplesNot built yet
Derive a column from typed examples of the output
Change a column's typeRuns as a SQL step
Change data type, detect type, set metadata or role (2 variants with a page)
Merge near-duplicate valuesRuns as a SQL step
Cluster near-duplicate values within one column and merge them (2 variants with a page)
Add a column by conditionRuns as a SQL step
If, case or switch expression to a column
Add a calculated columnRuns as a SQL step
New column from a formula or expression (5 variants with a page)
Encode categories as numbersRuns as a SQL step
One-hot, dummy, label or ordinal encoding (3 variants with a page)
Hash or mask a columnRuns as a SQL step
Hash, checksum, mask, encrypt, anonymise (4 variants with a page)
Promote a row to headersRuns as a SQL step
Promote a row to headers, or demote headers to a row (1 variant with a page)
Compare and combine conditionsRuns as a SQL step
Comparison and boolean expressions
Apply a maths functionRuns as a SQL step
Numeric functions: arithmetic, rounding, trigonometry (4 variants with a page)
Combine columns into oneRuns as a SQL step
Concatenate several columns into one
Fill in missing valuesRuns as a SQL step
Replace, coalesce, impute or flag missing values (4 variants with a page)
Parse a date or timeRuns as a SQL step
Parse, format or decompose dates and times; date arithmetic; time zones (8 variants with a page)
Parse nested valuesRuns as a SQL step
Parse or build JSON, XML, URL, key-value, nested structs and lists (9 variants with a page)
Replace valuesRuns as a SQL step
Replace or map values, find and replace, value mapper (2 variants with a page)
Match with a regular expressionRuns as a SQL step
Regex extract, match, replace (2 variants with a page)
Rescale a number columnRuns as a SQL step
Normalise, standardise, z-score, min-max, log transform
Fit columns to a layoutRuns as a SQL step
Fit columns to a target layout
Choose and rename columnsRuns as a SQL step
Keep, drop, reorder or rename columns (5 variants with a page)
Split a columnRuns as a SQL step
Split one column into several by delimiter, position or pattern (2 variants with a page)
Standardise addresses, names and phone numbersRuns as a SQL step
Parse and standardise addresses, names, phone numbers
Clean up textRuns as a SQL step
String functions: trim, case, pad, substring, remove characters, normalise whitespace or unicode (8 variants with a page)
Browse rowsA feature, not a step
View, preview or browse rows
Edit cells by handRuns as a SQL step
Change individual cells by hand, insert or delete specific rows (1 variant with a page)
Fill down or upRuns as a SQL step
Fill down or up across rows, and the inverse, blank down
Number or rank rowsRuns as a SQL step
Row number, record id, rank, dense rank, ntile (5 variants with a page)
Mark rowsNot built yet
Mark rows without removing them: select, flag, star
Sort rowsRuns as a SQL step
Order rows
Run a SQL queryRuns as a SQL step
Run a SQL query over the table
Calculate across rowsRuns as a SQL step
Running total, lag, lead, moving average, multi-row formula (6 variants with a page)
Fewer rows, same columns
Deduplicate rowsRuns as a SQL step
Distinct rows, unique by key, keep first or last per key (3 variants with a page)
Filter rowsRuns as a SQL step
Keep or drop rows by a condition (5 variants with a page)
Take the first or last rowsRuns as a SQL step
First N, last N, top N, row range, skip rows (3 variants with a page)
Sample rowsRuns as a SQL step
Random or stratified sample, train and test split (3 variants with a page)
Split a tableRuns as a SQL step
Split one table into several by a key or a condition
More rows, each traceable to one input row
Split a list into rowsRuns as a SQL step
One row to many: unnest, explode, split to rows (2 variants with a page)
A new grain: each row stands for a group of input rows
Summarise by groupRuns as a SQL step
Group by and summarise (4 variants with a page)
Count valuesRuns as a SQL step
Frequency table, value counts, count by
Gather rows into columnsRuns as a SQL step
Flatten a row group into columns, or spread columns back to rows
Add subtotalsRuns as a SQL step
Rollup, cube, grouping sets, subtotals
The same cells, rearranged
Pivot a tableRuns as a SQL step
Rows to columns, cross-tab
Transpose a tableRuns as a SQL step
Swap rows and columns
Unpivot a tableRuns as a SQL step
Columns to rows, melt
A table describing the input rather than holding its rows
Chart the dataA feature, not a step
Plot the data for exploration (2 variants with a page)
Measure correlationRuns as a SQL step
Correlation or association matrix, contingency table (2 variants with a page)
See a column's distributionRuns as a SQL step
Histogram, quantiles of one column
Profile a tableRuns as a SQL step
Summary statistics, distinct counts, null counts
List columns and typesRuns as a SQL step
List columns, types, metadata (3 variants with a page)
Check data qualityRuns as a SQL step
Data-quality rule, assertion, test, constraint or schema check (8 variants with a page)