Model operations
Operations that analyse the data or learn from it. Among them are statistical tests and regressions, clustering and outlier detection, training, scoring and evaluating models, forecasting a time series, and working with text, networks, geography and images. Some return the table with columns added, some a table describing the input, and some a trained model or a file.
37 operations, grouped by the shape of the result, each with its status in arcform today.
One row out per row in, with columns added or changed
Find outliersRuns as a SQL step
Outlier or anomaly detection
Cluster rowsNot built yet
K-means, hierarchical, DBSCAN, other clustering of rows (1 variant with a page)
Build features for a modelRuns as a SQL step
Generate features or row weights for a model
Work with geographyRuns as a SQL step
Geocode, spatial operations on geometry, grid cells, routing (7 variants with a page)
Reduce dimensionsNot built yet
PCA, SVD, UMAP, t-SNE, other dimensionality reduction
Classify textNot built yet
Classify text into categories
Embed text as vectorsRuns as a SQL step
Text or image to vector
Extract meaning from textNot built yet
Sentiment, named entities, topics, language detection (2 variants with a page)
Ask a language modelNot built yet
Call a large language model (2 variants with a page)
Read text and objects in imagesNot built yet
Image transformation, recognition, OCR, barcodes (1 variant with a page)
Fewer rows, same columns
Search text by relevanceRuns as a SQL step
Full-text search ranked by relevance
More rows, each traceable to one input row
Split text into wordsRuns as a SQL step
Tokenise, stem, n-grams, stop words, TF-IDF (4 variants with a page)
Forecast a time seriesNot built yet
ARIMA, ETS, other forecasting
A new grain: each row stands for a group of input rows
Find items that occur togetherRuns as a SQL step
Association rules, frequent itemsets, market basket
Analyse a networkRuns as a SQL step
Network or graph analysis (1 variant with a page)
Resample a time seriesRuns as a SQL step
Resample, decompose, difference, smooth, fill time gaps (3 variants with a page)
One table from two or more
Find similar rowsRuns as a SQL step
Row-to-row distance, nearest-neighbour search, vector search (2 variants with a page)
A table describing the input rather than holding its rows
Evaluate a modelRuns as a SQL step
Performance metrics, confusion matrix, ROC, lift, cross-validation (3 variants with a page)
Explain a modelNot built yet
Variable importance, partial dependence, SHAP, model inspection (3 variants with a page)
Optimise a planNot built yet
Linear programming, simulation, prescriptive optimisation
Use a probability distributionNot built yet
Distribution functions, density estimation (1 variant with a page)
Fit a regressionRuns as a SQL step
Linear, logistic or generalised linear regression as statistics (1 variant with a page)
Run a hypothesis testNot built yet
Hypothesis test, ANOVA, chi-square, t-test (1 variant with a page)
A model or a file, not a table
Calibrate a model's confidenceNot built yet
Rescale a model's confidences
Save or load a modelNot built yet
Store, load, export or import a trained model
Recommend itemsNot built yet
Collaborative filtering, matrix completion
Model a sequenceNot built yet
Hidden Markov and other sequence models, reinforcement learning
Model time to an eventNot built yet
Time-to-event models
Combine modelsNot built yet
Bagging, boosting, stacking, voting, a choice of learners, AutoML (1 variant with a page)
Train a linear modelNot built yet
Linear models, SVM, naive Bayes trained as a predictor (2 variants with a page)
Train a neural networkNot built yet
Neural network, deep learning
Learn rules from dataNot built yet
Rule induction, subgroup discovery
Train a tree modelNot built yet
Decision tree, random forest, gradient boosted trees (2 variants with a page)
Tune a modelNot built yet
Hyperparameter search, feature selection, feature weighting (2 variants with a page)
A model or file and a table in, the table out with columns added
Score with a modelNot built yet
Score or predict with a trained model (1 variant with a page)
Classify by nearest neighboursRuns as a SQL step
K-nearest neighbours
Not a data operation
Assemble a modelling pipelineA feature, not a step
Group models, assemble a modelling pipeline