# Embed text as vectors

Model / text

text or image to vector

Page: <https://meridian.online/operations/embed-text-as-vectors>

## Status

Runs as a SQL step.

arcform carries it as a `sql:` step today, with the `subtoken` extension.

## The DuckDB step

embed text and rank by cosine similarity

It reads the `orders` table the proofs share. Create it first:

```sql
CREATE TABLE orders AS SELECT * FROM (VALUES
  (1,  'Alice Ng',      'north', DATE '2026-01-03', 120.00, 'widget', 'a|b',   '{"channel":"web","items":[1,2]}'),
  (2,  'alice ng',      'north', DATE '2026-01-05',  80.50, 'gadget', 'b',     '{"channel":"shop","items":[3]}'),
  (3,  'Bob  Okafor',   'south', DATE '2026-01-05', 300.00, 'widget', 'a|c',   '{"channel":"web","items":[4,5,6]}'),
  (4,  'Carla Díaz',    'east',  DATE '2026-01-09',  NULL,  'gizmo',  NULL,    '{"channel":"web","items":[]}'),
  (5,  'Bob Okafor',    'south', DATE '2026-01-12',  45.25, 'gadget', 'c',     '{"channel":"phone","items":[7]}'),
  (6,  'Dev Patel',     'west',  DATE '2026-01-12', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (7,  'Carla Diaz',    'east',  DATE '2026-01-20',  99.99, 'gizmo',  'b|c',   '{"channel":"shop","items":[10]}'),
  (8,  'Dev Patel',     'west',  DATE '2026-02-02', 510.00, 'widget', 'a',     '{"channel":"web","items":[8,9]}'),
  (9,  'Eve Larsen',    NULL,    DATE '2026-02-03',  15.00, 'gadget', NULL,    '{"channel":"web","items":[11]}'),
  (10, 'Alice Ng',      'north', DATE '2026-02-10', 220.00, 'gizmo',  'a|b|c', '{"channel":"phone","items":[12,13]}')
) t(order_id, customer, region, order_date, amount, product, tags, meta);
```

Then run the statement:

```sql
INSTALL subtoken FROM community; LOAD subtoken;
CREATE TEMP TABLE result AS
WITH e AS (SELECT order_id, product, customer, subtoken_embed(customer || ' bought a ' || product) AS v FROM orders),
     q AS (SELECT subtoken_embed('someone purchased a gadget') AS v)
SELECT e.order_id, e.customer, e.product, len(e.v) AS dim, list_cosine_similarity(e.v, q.v) AS cos FROM e, q;
SELECT customer, product, round(cos, 4) AS cos FROM result ORDER BY cos DESC LIMIT 3;
SELECT CASE WHEN ((SELECT min(dim) = 256 AND max(dim) = 256 AND arg_max(product, cos) = 'gadget' AND min(cos) FILTER (WHERE product = 'gadget') > max(cos) FILTER (WHERE product <> 'gadget') FROM result)) THEN 'as expected' ELSE error('the orders for a gadget did not rank above the others for a query about a gadget') END AS verdict;
```

It needs the `subtoken` extension.

## What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

- Duckle: [`Embeddings`](https://github.com/slothflowlabs/duckle/blob/f85f8960edfb6efbf4cb89d58a7b32e16df0e867/website/docs/components.html) (read 2026-09-25)
- Apache Hop: [`Embed text`](https://hop.apache.org/manual/next/pipeline/transforms/embedtext.html) (read 2026-09-27)
- KNIME Analytics Platform: [`KNIME Textprocessing - Deeplearning4J Integration (64bit only) (legacy) (extension)`](https://api.hub.knime.com/extensions/org.knime.features.ext.textprocessing.dl4j) (read 2026-09-27)
- RapidMiner Studio: [`Embeddings (FastEmbed)`](https://docs.rapidminer.com/latest/studio/operators/extensions/Generative%20Models/Embeddings/embeddings_fastembed.html) (read 2026-09-27)
- RapidMiner Studio: [`Embeddings (OpenAI)`](https://docs.rapidminer.com/latest/studio/operators/extensions/Generative%20Models/Embeddings/embeddings_openai.html) (read 2026-09-27)
- Apache SeaTunnel: [`Embedding`](https://seatunnel.apache.org/docs/2.3.13/transforms/embedding) (read 2026-09-27)
- subtoken: [`subtoken_embed`](https://github.com/meridian-online/subtoken/blob/ceb61bd796b15b4d54ca3bb2db8a74792d67026e/README.md) (read 2026-09-27)
- Talend: [`tEmbeddingAI`](https://help.qlik.com/talend/en-US/components/8.0/ai/ai-component) (read 2026-09-27)