Shape / inspect · All operations

Validate files against a schema

A variant of Check data quality

Check that XML, JSON or CSV content or files are well-formed and conform to a schema such as XSD, DTD or JSON Schema.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Apache Hop (4)
Apache NiFi (3)
  • ValidateCsvread 2026-09-27

    Validate CSV content, or an attribute value, against a user-specified CSV schema

  • ValidateJsonread 2026-09-27

    Validate JSON content against a JSON Schema

  • ValidateXmlread 2026-09-27

    Validate XML held in content or in a named attribute

AWS Glue (4)
  • FileFreshnessread 2026-09-27

    Check how recent the files at an Amazon S3 path are

  • FileMatchread 2026-09-27

    Check by checksum that a file matches a checksum or another file

  • FileSizeread 2026-09-27

    Check the size of files against a condition

  • FileUniquenessread 2026-09-27

    Check by checksum that files are unique

KNIME Analytics Platform (1)
Pentaho Data Integration (1)
  • XSD Validatorread 2026-09-27

    Validate XML in files or fields against an XML Schema Definition

Talend (3)
  • tDTDValidatorread 2026-09-27

    Validate an XML file against a DTD and output the validation log

  • tJSONValidatorread 2026-09-27

    Validate JSON data against a JSON schema

  • tXSDValidatorread 2026-09-27

    Validate an XML file or flow against an XSD and output the validation log

Background

A valid Extensible Markup Language (XML) document is a well-formed XML document that also conforms to the rules of a Document Type Definition (DTD).[1] JSON Schema is a declarative language for defining the structure and constraints of JavaScript Object Notation (JSON) data.[2] XML Schema Definition (XSD) specifies how to formally describe the elements of an XML document, and can express rules a document must meet to be valid.[3]

When to use it

An XML processor that encounters a violation of the well-formedness rules is required to report the error and to cease normal processing.[1] Check that a file is well formed before checking it against a schema, so the report can say whether the file could not be read at all or was read and did not conform. (Meridian's view)

In DuckDB, the comma-separated values (CSV) reader detects structural errors including a value that cannot be cast to the expected type, a line with too few columns and a line with too many.[4] By default DuckDB's CSV reader stops at the first structural error, and with store_rejects set it skips faulty lines and stores them in rejects tables.[4] DuckDB's CSV reader is affected by projection pushdown, so a query that selects only some columns does not meet cast errors in the columns it leaves out.[4] When the point is to validate a CSV file, read every column it should have, not only the ones the next step needs. (Meridian's view)

In JSON Schema, leaving out properties is valid by default and so is providing additional properties, unless the required and additionalProperties keywords say otherwise.[5] List every field that must be present under required, and decide on purpose whether a document may carry fields the schema does not name. (Meridian's view)

The method

A well-formed XML document has begin, end and empty-element tags that are correctly nested, with none missing and none overlapping, and a single root element.[1] XML Schema was published as a World Wide Web Consortium (W3C) recommendation in May 2001, the first separate schema language for XML to achieve Recommendation status.[3] XSD offers namespace awareness and datatypes, such as integers and dates, which XML's native Document Type Definitions do not.[3] XSD 1.1 became a W3C Recommendation in April 2012.[3]

JSON Schema uses a set of keywords to define the properties of data, and validating a JSON instance against a schema requires a validator that implements the JSON Schema specification.[2]

Sources

  1. [1]Well-formed document - Wikipediasecondary source
  2. [2]JSON Schema - What is JSON Schema?documentation
  3. [3]XML Schema (W3C) - Wikipediasecondary source
  4. [4]Reading Faulty CSV Files – DuckDBdocumentation
  5. [5]JSON Schema - objectdocumentation