Skip to content
Direction Draft

Source-aligned data as the governed foundation

Every layer starts again

A payment record enters the analytical estate. The analytical model decides who owns it and what "settled" means. The semantic layer decides again. The report that uses it makes its own call about whether it contains personal data.

Each layer had good reasons. None of them was the domain that actually knows what a payment is.

Several definitions of one thing

  • The same concept ends up with several independently maintained definitions that slowly diverge.
  • Classification is re-decided, so personal data is protected in one layer and not in the next.
  • Nobody can say which layer's version is authoritative, because each one looks authoritative from inside.

Established once, inherited downstream

Business meaning should enter the ecosystem close to the producing domain, at the source-aligned data product. Downstream products and layers should inherit its ownership, meaning, contract and classification, not re-establish them.

Reshaping data downstream is legitimate and often necessary. What this constrains is the re-derivation of governance, not of shape.

The picture

Established once, inherited downstream Top row, the direction: a source-aligned payment product carries four stamps, owner, meaning, contract and classification, established once. They travel unchanged to a daily analytical fact and on to a semantic view, even though each box changes shape. Bottom row, the anti-pattern: each layer draws its own stamps, re-decided three times. Direction · established once, inherited Source-aligned payment product OWNER MEANING CONTRACT CLASS Daily payment fact reshaped for analysis SAME FOUR · INHERITED Semantic view INHERITED Anti-pattern · re-decided at every layer Source tableowner? meaning₁ Analytical modelowner₂ meaning₂ Reportmeaning₃ Shape may change downstream. Ownership, meaning, contract and classification should not.
The boxes change shape as data moves downstream. The four stamps do not.

A payment, reshaped but not redefined

Illustrative example

The payments domain publishes a source-aligned payment product. It names its owner, defines what a settled payment is, publishes a contract and classifies the product as containing personal data.

The finance team reshapes it into a daily payment fact for reporting: a different grain and a different table. It keeps the identifiers, references the contract and inherits the classification. When the payments domain refines its definition of "settled", the change flows through, and nobody has to find the three other places it was re-decided.

In detail

What may be added downstream

Downstream products may add their own derived meaning, such as conformed dimensions or certified metrics, as long as it stays traceable to the foundation it derives from.

A point of entry, not a container

A source-aligned product is the point of entry for the meaning it produces. It is not a container for the whole enterprise's semantics, and it does not own the enterprise ontology.

Not this

  • Not a ban on dimensional modelling or analytical reshaping.
  • Not a technology choice for source-aligned products, contracts or classification.

Unresolved

  • How is an inherited classification tightened (never silently loosened) when a downstream combination makes data more sensitive than any of its sources?

Join the argument

Agree, challenge or add evidence

Architecture improves when its assumptions are challenged. Say which kind of contribution you're making:

  • Challenge I disagree because…
  • Evidence We've seen this too…
  • Question How would this work when…?
  • Alternative Another way to approach this…
  • Extension This also implies…

Please keep clients, employers and colleagues unidentifiable. Strong contributions may be quoted and credited on this site. Read the contribution guidelines and the privacy notice.

Comments are provided by giscus and stored on GitHub. Posting requires a GitHub account.

Next in the argument Quality starts at the producer Every consumer repairs the same quality problems in its own way, so trust has to be rebuilt many times over. Make semantic context explicit

Connected thinking

Principle · draft Meaning and accountability belong at the source Domain owners maintain the meaning of what they publish, and reconciling meaning across contexts is an explicit, recorded decision. Concept · evolving Data product Principle · draft Semantic authority is independent of consumption Business concepts and metrics need governed identities that exist independently of every tool that consumes them. Pattern · proposed Composable data product ecosystem