Skip to content
Principle Draft

Quality starts at the producer

Three consumers, three repairs

Shipment events arrive late, sometimes twice, occasionally not at all. Finance removes duplicates and waits an hour before closing the day. Operations removes duplicates and waits four. Customer service fills the gaps from another system.

Each fix is sensible. Together they produce three versions of how many parcels shipped yesterday.

Trust manufactured downstream

  • The same repair is built, paid for and maintained by every consumer, and the copies drift.
  • Nobody upstream knows the problem exists, because the consumers absorbed it.
  • A new consumer inherits none of the fixes and starts again.

Produce trust instead of repairing it

Produce trustworthy data rather than expecting consumers to manufacture trust. The producer defines and observes measurable quality expectations (timeliness, completeness, accuracy, availability) as close as practical to where the business fact is created, and publishes the evidence alongside the data.

The picture

Quality repaired downstream versus quality produced at source Top: a producer publishes shipment events and three consumers each build their own repair: deduplicate, wait for late events, fill gaps, each slightly different. Bottom: the producer declares and observes quality expectations, timeliness, completeness, accuracy and availability, and publishes the evidence; the three consumers read the evidence instead of repairing. Trust manufactured by every consumer Producer shipment events fix A: dedupe, wait 1h fix B: dedupe, wait 4h fix C: fill gaps Finance Operations Customer service three truths Trust produced once, at the source Producer DECLARED AND OBSERVED timeliness · completeness accuracy · availability evidence published with the data Finance Operations Customer service one truth, checked
Quality expectations declared and observed once, at the source, and read by every consumer.

Shipment events, with evidence

Illustrative example

The team that owns shipping declares its expectations: events are unique per parcel and status, arrive within fifteen minutes, and cover every dispatched order. It observes them continuously and publishes the results with the product.

Finance, operations and customer service stop maintaining their own repairs. When completeness drops one morning, the producer sees it first, and every consumer can see the same evidence rather than discovering it separately in three different reports.

In detail

Measurable, or it is only a hope

A quality expectation that cannot be observed is little more than documentation. Expectations should be specific enough to test, and their results visible to consumers. That thread continues in accepted commitments must be verifiable.

The producer's quality, not every consumer's

The producer is responsible for the quality of what it publishes. A consumer with stricter or different needs for its own purpose states them. It does not expect the producer to guess.

Not this

  • Not a promise of perfect data. It is a promise of known, observed quality.
  • Not a reason for consumers to stop checking what matters to their own use.

Unresolved

  • How are quality expectations set when a producer has many consumers with different tolerances?

Join the argument

Agree, challenge or add evidence

Architecture improves when its assumptions are challenged. Say which kind of contribution you're making:

  • Challenge I disagree because…
  • Evidence We've seen this too…
  • Question How would this work when…?
  • Alternative Another way to approach this…
  • Extension This also implies…

Please keep clients, employers and colleagues unidentifiable. Strong contributions may be quoted and credited on this site. Read the contribution guidelines and the privacy notice.

Comments are provided by giscus and stored on GitHub. Posting requires a GitHub account.

Next in the argument Metadata is produced, not reconstructed A catalogue tries to work out ownership, meaning and lineage months after the data was produced, and is always out of date. Source-aligned data as the governed foundation

Connected thinking

Principle · draft Meaning and accountability belong at the source Domain owners maintain the meaning of what they publish, and reconciling meaning across contexts is an explicit, recorded decision. Principle · draft Metadata is produced, not reconstructed Metadata should flow from declaring, building, deploying and operating a product, with people adding only what needs judgement. Principle · proposed Accepted commitments must be verifiable Concept · evolving Data contract