Quick Start

Every analyst knows the feeling: you open a CSV and every column is a string. Is that column of numbers a price or a postal code? Are the dates ISO 8601 or DD/MM/YYYY? Is that blank cell a missing value or an empty string?

This guide walks you through the full Finetype pipeline — from raw CSV to a typed, validated DuckDB table — in five steps. Along the way, you will catch real data quality issues that would otherwise surface as silent errors downstream.

Prerequisites

Finetype and DuckDB run on macOS, Linux, and Windows.

Install DuckDB

brew install duckdb

sudo apt install duckdb

winget install DuckDB.cli

curl https://install.duckdb.org | sh

Install Finetype

curl -fsSL https://install.meridian.online/finetype | bash

curl -fsSL https://install.meridian.online/finetype | bash -s -- <version_tag>

brew install meridian-online/tap/finetype

cargo install finetype-cli

irm https://install.meridian.online/finetype/win | iex

Verify both are working:

duckdb --version
finetype --version

The Sample Dataset

We will use a small contacts dataset with realistic quality issues — the kind you find in every export from a CRM, spreadsheet, or third-party API.

Download the file directly: contacts.csv

Or copy and save it locally:

id,name,email,created_at,ip_address,amount
1,Alice Chen,alice@example.com,2024-01-15T09:30:00Z,192.168.1.10,149.99
2,Bob Smith,bob@company.org,2024-02-20T14:15:00Z,10.0.0.42,2500.00
3,Carol Wu,carol@startup.io,2024-03-08T11:00:00Z,172.16.0.1,89.50
4,Dan Reeves,dan@university.edu,2024-04-12T16:45:00Z,192.168.0.5,1200.00
5,Eve Nakamura,eve@research.net,2024-05-01T08:00:00Z,10.10.10.1,340.75
6,Frank Osei,not-an-email,2024-06-15T10:20:00Z,203.0.113.7,75.00
7,Grace Patel,grace@design.studio,2024-07-15T18:30:00Z,10.0.0.99,1850.25
8,Hiro Tanaka,hiro@tech.co,2024-08-03T13:45:00Z,,425.00
9,Isla Morgan,isla@gov.uk,2024-09-19T07:30:00Z,192.168.1.200,N/A
10,Jin Li,jin@startup.io,2024-10-05T16:00:00Z,999.999.999.999,3200.00
11,Kara Novak,kara@example.com,2024-11-11T09:00:00Z,10.0.1.50,620.00
12,Leo Santos,leo@agency.com,2024-12-01T12:00:00Z,172.16.5.20,

This looks tidy at first glance, but there are problems hiding in plain sight:

• Row 6: not-an-email is not a valid email address
• Row 8: missing IP address
• Row 9: N/A in the amount column — not a number
• Row 10: 999.999.999.999 is not a valid IP address
• Row 12: missing amount

DuckDB will happily read this file and load everything as VARCHAR. The problems only surface later, when a CAST fails or an aggregate silently ignores nulls. Finetype catches these issues upfront — and, as you will see, draws a clear line between a value that is missing (rows 8 and 12) and a value that is wrong (rows 6, 9, and 10).

Step 1: Classify Individual Values

Before profiling the whole file, build intuition for how Finetype thinks. The infer command classifies text. Give it the first email address in the file:

finetype infer -i "alice@example.com" --confidence

identity.person.email
  confidence: 0.9900 (1 samples)
  disambiguation: deterministic_fast_path

finetype infer -i "192.168.1.10" --confidence

technology.internet.ip_v4
  confidence: 0.9900 (1 samples)
  disambiguation: deterministic_fast_path

Each value gets a three-part label: domain.category.type. An email address and an IP address each have a shape distinctive enough that Finetype settles them from the value alone. That is what deterministic_fast_path says, and the confidence is then a fixed 0.99. infer settles a lone timestamp from created_at, such as 2024-01-15T09:30:00Z, the same way.

Not every value settles on its own. 03/04/2024 could be 3 April or 4 March, and nothing in the value says which:

finetype infer -i "03/04/2024" --confidence

datetime.date.mdy_slash
  confidence: 0.7453 (1 samples)

With no rule to settle it, the model makes its best guess: month-first, at a confidence well short of the email's. Give it a column instead, with a second date whose middle field is 15. There is no fifteenth month, so the column must be month-first:

printf '03/04/2024\n03/15/2024\n' > dates.txt
finetype infer -f dates.txt --confidence

datetime.date.mdy_slash
  confidence: 0.9900 (2 samples)
  disambiguation: deterministic_fast_path

With no header given, infer settles the column on the same structural fast path, at the same fixed 0.99. profile doesn't take that path: it runs every column through the model with the column name as a hint, so the slash-date rule picks the type and CONF stays the model's confidence, as the overview's order_date example shows.

This is why the next step profiles the whole file: the other values in a column, and its name, settle what one value can't. Day-first and month-first dates explains how Finetype tells the two apart.

A bad value never crashes — Finetype classifies what it sees:

finetype infer -i "not-an-email" --confidence

representation.text.entity_name
  confidence: 0.4900 (1 samples)

not-an-email reads as a name of some kind, and the low confidence is the tell: when a value matches no sharp pattern, the score drops. Hold that thought: this value is row 6, and you will meet it again in the email column's confidence in Step 2 and as a reject in Step 4.

Step 2: Profile the Dataset

The profile command analyses every column and picks the best semantic type for each:

finetype profile -f contacts.csv

profile and validate also print progress lines, such as Read 12 rows, to stderr; the outputs below leave them out.

FineType Column Profile — "contacts.csv" (12 rows, 6 columns)
════════════════════════════════════════════════════════════════════════════════

  COLUMN                    TYPE                                      BROAD   CONF
  ──────────────────────────────────────────────────────────────────────────────
  id                        representation.identifier.increment      BIGINT  98.5% [numeric_sequential_detection]
  name                      identity.person.full_name               VARCHAR  96.0%
  email                     identity.person.email                   VARCHAR  63.8% ⚑ low
  created_at                datetime.timestamp.iso_8601            TIMESTAMP 100.0%
  ip_address                technology.internet.ip_v4               VARCHAR 100.0% [ipv4_detection]
  amount                    representation.numeric.decimal_number    DOUBLE  60.0% [amount_bare_number_veto:amount] ⚑ low

6/6 columns typed, 12 rows analyzed

A few things to notice:

• BROAD is the DuckDB storage type Finetype recommends. VARCHAR columns stay as strings (emails and IPs are semantically typed but stored as text); created_at becomes TIMESTAMP; amount becomes DOUBLE.
• CONF is the column-level confidence, followed by the quality band where it is not high. Two columns read ⚑ low, and a low band says the type may be wrong: check the column. The Finetype overview explains the bands.
• email reads low because of row 6. The type is right, and not-an-email, the value that read low on its own in Step 1, is what pulls the column's confidence down: profile the file without row 6 and email reads 100.0%. The profiler tells you the intended type; catching the rows that violate it is the job of validation in Step 4.
• amount reads low for another reason. Its name suggested a currency amount, but its values carry no currency symbol, so a veto, amount_bare_number_veto in brackets, turned it back into a plain decimal number and capped its confidence at 0.6. If they are money, the file doesn't say in which currency, so this guide keeps them as plain numbers.
• The other bracketed names show which rule fired: numeric_sequential_detection spotted the running id, and ipv4_detection matched the address pattern.

Finetype has now told you what each column should be. The next step is to capture that as a formal schema.

Step 3: Generate a Schema

Run profile with -o json-schema to turn the profile results into a JSON Schema that describes the expected shape and types of your data:

finetype profile -f contacts.csv -o json-schema

{
  "$id": "finetype://contacts.csv",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "amount": {
      "pattern": "^-?[0-9]+(\\.[0-9]+)?([eE][+-]?[0-9]+)?$",
      "type": "string",
      "x-finetype-label": "representation.numeric.decimal_number",
      "x-finetype-pii": false
    },
    "created_at": {
      "maxLength": 27,
      "minLength": 20,
      "pattern": "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(\\.\\d{1,6})?Z$",
      "type": "string",
      "x-finetype-label": "datetime.timestamp.iso_8601",
      "x-finetype-pii": false
    },
    "email": {
      "maxLength": 254,
      "minLength": 5,
      "pattern": "^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$",
      "type": "string",
      "x-finetype-label": "identity.person.email",
      "x-finetype-pii": true
    },
    "id": {
      "pattern": "^[0-9]+$",
      "type": "string",
      "x-finetype-label": "representation.identifier.increment",
      "x-finetype-pii": false
    },
    "ip_address": {
      "pattern": "^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$",
      "type": "string",
      "x-finetype-label": "technology.internet.ip_v4",
      "x-finetype-pii": false
    },
    "name": {
      "maxLength": 200,
      "minLength": 2,
      "pattern": "^[\\p{L}\\s'\\-.,]+$",
      "type": "string",
      "x-finetype-label": "identity.person.full_name",
      "x-finetype-pii": true
    }
  },
  "required": [
    "created_at",
    "email",
    "id",
    "name"
  ],
  "title": "contacts",
  "type": "object"
}

Save this to a file — it becomes the contract between your raw data and your typed tables:

finetype profile -f contacts.csv -o json-schema > schema.json

Each property carries a pattern (the regex every value must match), an x-finetype-label (the semantic type), and an x-finetype-pii flag marking columns that hold personal data — here, name and email. Standard JSON Schema validators understand the pattern; the x-finetype-* extensions are ignored by other tools but drive Finetype's own validation in the next step.

Note the required array: it lists only created_at, email, id, and name — the columns with no empty cells. Finetype saw blanks in ip_address and amount (rows 8 and 12) and left them optional, so a missing value there is allowed rather than flagged.

Step 4: Validate and Materialise

Now that you have a schema, validate every row against it. By default validate runs check-only and reports how many rows pass:

finetype validate contacts.csv schema.json

Validation Report
════════════════════════════════════════════════════════════
  Input:        contacts.csv
  Schema:       schema.json
  Mode:         check-only (no .db written)

  Total rows:            12
  Valid rows:             9
  Invalid rows:           3
  Rejects:                3
  Grade:             C
════════════════════════════════════════════════════════════

Nine of the twelve rows pass; three are rejected, earning the dataset a Grade of C. The exit code tells a CI pipeline what happened: 0 = no rejects, 1 = rejects present, 2 = error.

To gate the data and cast the valid rows into a typed DuckDB table in the same pass, add --db and --table:

finetype validate contacts.csv schema.json --db contacts.db --table contacts

This writes a typed contacts table (valid rows, per-column transforms applied) plus a finetype_reject_errors sidecar table capturing everything that failed. Inspect the rejects with DuckDB:

duckdb contacts.db -c "SELECT line, column_name, error_type, expected_type FROM finetype_reject_errors ORDER BY line;"

┌───────┬─────────────┬───────────────┬───────────────────────────────────────┐
│ line  │ column_name │  error_type   │             expected_type             │
│ int64 │   varchar   │    varchar    │                varchar                │
├───────┼─────────────┼───────────────┼───────────────────────────────────────┤
│     6 │ email       │ SEMANTIC_TYPE │ identity.person.email                 │
│     9 │ amount      │ SEMANTIC_TYPE │ representation.numeric.decimal_number │
│    10 │ ip_address  │ SEMANTIC_TYPE │ technology.internet.ip_v4             │
└───────┴─────────────┴───────────────┴───────────────────────────────────────┘

The three rejected rows are exactly the bad values we spotted earlier:

Rows 8 and 12 have missing values but are not rejected — ip_address and amount are optional, so a blank is a valid null. The distinction matters: a missing value is expected; N/A typed as a string where a number should be is a data quality error.

Step 5: Query the Typed Table

The --db/--table run in Step 4 already materialised a fully typed contacts table — no manual CAST statements, no guessing at date formats, no silent type coercion errors. Open it and query with proper types:

duckdb contacts.db

-- Query with proper types
SELECT name, amount, created_at
FROM contacts
WHERE created_at > TIMESTAMP '2024-06-01'
ORDER BY amount DESC;

┌─────────────┬─────────┬─────────────────────┐
│    name     │ amount  │     created_at      │
│   varchar   │ double  │      timestamp      │
├─────────────┼─────────┼─────────────────────┤
│ Grace Patel │ 1850.25 │ 2024-07-15 18:30:00 │
│ Kara Novak  │   620.0 │ 2024-11-11 09:00:00 │
│ Hiro Tanaka │   425.0 │ 2024-08-03 13:45:00 │
│ Leo Santos  │    NULL │ 2024-12-01 12:00:00 │
└─────────────┴─────────┴─────────────────────┘

Dates are TIMESTAMP, and amounts are DOUBLE, the plain decimal number Step 2 settled on — date arithmetic and numeric aggregation just work. The four rows are the post-June contacts; Leo Santos shows a NULL amount, the empty cell from row 12 carried through cleanly as a null rather than a parse error.

The full pipeline in four commands:

finetype infer -i "alice@example.com" --confidence                            # Build intuition
finetype profile -f contacts.csv                                              # Discover column types
finetype profile -f contacts.csv -o json-schema > schema.json                 # Capture the contract
finetype validate contacts.csv schema.json --db contacts.db --table contacts  # Validate & materialise

Bonus: DuckDB Extension

If you prefer to stay in SQL, the Finetype DuckDB extension lets you profile, validate, and classify directly in queries:

INSTALL finetype FROM community;
LOAD finetype;

-- Classify a single value
SELECT ft_infer('alice@example.com');
-- → identity.person.email

-- Or profile a whole table at once
FROM ft_profile('contacts');

See the DuckDB Extension docs for the full function reference.

What's Next?