Data quality

Data quality problems do not look like failures. The job succeeds, the row count is plausible, and the numbers are wrong - a silently renamed column, a source that started emitting duplicates, a filter that dropped a category. Catching that class of wrong requires assertions about the data, not just the process: the discipline splits into what you declare (contracts), what you test (checks), and what you watch (observability).

Data contracts: declare the shape before you check it

A data contract is the explicit agreement between producer and consumer about what the data is - schema (columns, types, nullability), semantics (what status = 'closed' means), and guarantees (freshness, completeness, allowed value ranges). Without one, “quality” is a vibes-based argument after the incident; with one, a breaking change is detectable at the boundary, before it lands in the warehouse.

A contract does not need to be a formal standard - a versioned schema plus a written statement of guarantees is a contract. What matters is that breaking it is loud: the producer’s schema change fails a compatibility check instead of silently altering what the consumer receives.

The checks that catch real damage

Quality checks fall into a few families, ordered roughly by how often they save you:

• Schema and integrity. Columns present with expected types; primary keys unique and not null; foreign keys resolving. In dbt these are the built-in unique, not_null, accepted_values, and relationships tests - cheap, fast, and the first thing to run on every load.
• Freshness. The data is only good if it is recent - a table whose max event time is three days old is a stalled pipeline wearing a healthy costume. Freshness checks compare the newest record to now, not the job’s last run status.
• Volume and completeness. Row counts within an expected band; every expected slice present (all regions, all dates). A run that loads 4% of the usual volume “succeeded” in every sense except the one that matters.
• Distribution. Values that drift - a column that was 2% null becoming 40% null, a category distribution shifting. These are the slow failures that threshold checks miss; they need statistical comparison over time, not fixed bounds.
• Reconciliation. The destination matches the source - row counts, aggregates, or checksums per slice. The only check that proves the pipe did not drop data in the middle.

Where checks live

Checks placed at the wrong point are either useless or blocking:

• At ingestion, validate before writing - a bad extract quarantined is a bad day; a bad extract loaded is a bad week. Idempotent overwrite of the raw slice means you can hold and re-run.
• In the transformation layer (dbt tests, or in-pipeline assertions), run on every build and make failures block or at least flag the model - the test suite is part of the pipeline, not a report someone reads later.
• At the boundary to consumers, contract checks on what leaves: a downstream-facing table that no longer matches its declared schema should fail loudly, not ship.

The orchestrator is where enforcement lives: a failed test that does not fail the run is documentation, not a gate. dbt build exists for exactly this - test results stop downstream models rather than producing a warning nobody sees.

Observability: the watching layer

Beyond per-run checks, data observability is the continuous version: tracking freshness, volume, and schema across tables over time and alerting on drift. It answers the question a single run cannot - “is this table still behaving like itself” - and surfaces the failures that fall between the checks: a table that is fresh but subtly wrong, a pipeline that is green but slower each day.

Failure modes and diagnostics

• The green-but-wrong pipeline. The signature failure - job status is not a quality metric. Freshness and volume checks on the data are what separate “ran” from “right”.
• Alert fatigue. Thresholds tuned tight enough to catch real problems fire on noise; teams learn to ignore them, and the system becomes decoration. Fewer, better checks on the tables that matter beats a thousand ignored warnings.
• Contract drift. The declared schema and the actual schema diverge because nothing enforces the contract - the check exists but is advisory. Enforcement at write time is the difference between a contract and a wish.
• Blame ambiguity. Without contracts, “bad data” incidents become an argument about whether the producer changed or the consumer assumed. The contract is what makes the question answerable.

What this is not for

Data quality tooling does not fix a bad source - it tells you the data is bad, and someone still has to decide what to do about it. It is also not a substitute for pipeline design: idempotent writes and bounded blast radius prevent quality incidents; checks detect the ones prevention missed. And a check that does not block is monitoring, not quality - if the number that matters can ship anyway, the gate is a suggestion.