A data mesh is a sociotechnical approach to analytical data that decentralizes responsibility to business domains while standardizing the way data is produced and consumed. It’s grounded in four principles: domain ownership, data as a product, a self‑serve data platform, and federated governance. In practice, it asks each domain team to publish data as a product—discoverable, trustworthy, and operable—while a common platform automates cross‑cutting rules (access, lineage, quality, security).
Zhamak Dehghani frames a data product as an architectural quantum: the smallest independently deployable unit that bundles data, code, metadata, and policy, with a versioned contract and a clear interface (APIs or governed views). Treating both foundational and derived products as quanta is the key to decoupled evolution without breaking interoperability.
Within that mesh, you’ll encounter two families of products:
- Foundational data products: stable, domain‑anchored building blocks (e.g., Well/Asset Master, Security Master, Positions & Trades). They emphasize identity, time bases, and authoritative facts.
- Derived data products: purpose‑built compositions or intelligence made from those blocks (e.g., Operations 360, Performance Attribution, Client‑Reporting Pack).
A useful way to recognize derived products is by four recurring composition patterns:
- Restricted subsets — governed slices for particular roles/regions.
- Composites — joins across products on conformed identifiers and time grains.
- Semantic/metric products — shared, governed definitions (e.g., “MTBF,” “Loss Ratio,” “Net Return after fees”).
- Inferred/feature products — ML features and model outputs packaged as products for reuse.
The art is keeping foundations thin and stable (IDs, clocks, taxonomies) and moving variability (joins, business semantics, metrics, ML) into derived products with explicit contracts. This balances local speed with mesh‑wide trust.
What every product needs (foundation or derived)
- Contract: schema + semantics, SLOs (freshness, completeness, availability), quality rules, and versioning.
- Interface: zero‑/low‑copy access (APIs, governed views), pagination/filters, and clear error semantics.
- Observability: lineage, data/metric SLIs, usage telemetry.
- Policy as code: access control, masking/row‑level rules, retention, and privacy/compliance controls—enforced by the platform as part of federated governance.
Deep Dive 1 — Oil & Gas: from wells to reliability decisions
Foundations (source‑aligned products)
- Well / Asset Master — canonical
asset_id, taxonomy, location, commissioning dates. - Production Telemetry — time‑series rates/pressures/temps with declared units and sampling cadence.
- Maintenance Work Orders — events, failure codes, parts, durations.
These are intentionally narrow, authoritative surfaces designed for predictable joins over identity and time.
Derived products (purpose‑built)
A) Operations 360 (Composite + Semantic/Metric)
- Goal: One operational view—production ⨝ downtime ⨝ maintenance backlog—at standard grains (1‑min, 15‑min, daily).
- Inputs: Well/Asset Master ⨝ Production Telemetry ⨝ Work Orders on
asset_id, aligned clocks/calendars. - Interface:
- Read‑only governed view and query API;
- Metric catalog for MTBF/MTTR and deferment definitions;
- Lineage from raw signals → engineered features → published metrics.
- Contract highlights: P95 freshness ≤ 5 minutes from SCADA arrival (plus a daily batch tier); ≥99% interval completeness for key rates; physical‑range and monotonic checks; semver with 90‑day deprecation.
B) Downtime Risk (Inferred/Features)
- Goal: Predict near‑term failure windows (e.g., 24–72 hours) and surface drivers.
- Inputs: Rolling statistics from telemetry (variance, kurtosis, lag deltas), recent work‑orders, environmental signals.
- Interface:
- Feature product with training/serving parity and freshness SLOs;
- Score product exposing
asset_id,timestamp,risk_score,ttl, and top contributing factors.
- Governance: Model card (training window, validation metrics, blind spots), explanations, and read‑only consumption endpoints.
Why mesh‑native? Ownership stays with domain teams; composition is by contract (IDs, time bases, taxonomies), and enforcement (access, lineage, SLO checks) is automated at product boundaries—exactly what the self‑serve platform and federated governance principles prescribe.
Deep Dive 2 — Wealth Management: from books & records to client‑ready insights
Foundations (source‑aligned products)
- Security Master — canonical instrument identifiers (ISIN/CUSIP/Sedol), classifications (GICS, country, sector), corporate‑action rules.
- Positions & Trades — holdings and transactions with lot‑level detail and timestamps.
- Calendars & FX — trading/settlement calendars and rates; benchmark hierarchies.
These products are stable cores used across portfolio accounting, risk, and reporting.
Derived products
A) Exposure & Performance Attribution (Composite + Semantic/Metric)
- What it is: A governed, audit‑ready product that joins Positions & Trades with Security Master, benchmark classifications, calendars, and FX to expose exposures and returns at multiple cuts (asset class, sector, region, factor).
- Attribution: Supports arithmetic Brinson (allocation/selection/interaction) and optionally risk‑/factor‑based attribution. Brinson methods remain the common equity attribution baseline; risk‑based approaches extend this to explain contributions via factors.
- Contract highlights:
- Freshness: end‑of‑day T+0 with EOD NAV; optional intraday snapshots.
- Completeness: ≥ 99.5% instrument coverage with classification mappings.
- Policy: benchmark/FX sources documented; restatements tracked with lineage; calculation dictionary for returns (time‑weighted vs. money‑weighted) and contribution math.
B) Client‑Reporting Pack (Restricted Subsets + Semantic/Metric)
- What it is: Pre‑composed, entitlement‑aware slices (by client/account/strategy) that standardize ex‑post costs & charges and performance disclosures.
- Compliance anchors:
- GIPS® alignment for fair representation and full disclosure of performance;
- MiFID II guidance for ex‑post costs and charges presentation (fair, clear, not misleading).
- Interface:
- API and governed views with row/column‑level security;
- Parameterization for reporting period, base currency, benchmark set;
- Artifact generation (PDF/Excel) via platform jobs using the same data contract.
- Contract highlights:
- Freshness: monthly packet by T+5 business days; ad hoc on demand;
- Quality: reconciliation checks against books & records/NAV; break thresholds and attestation workflow;
- Compliance metadata: disclosure text versions, fee schedules, benchmark changes, and audit trail.
C) Factor Exposures & Risk Signals (Inferred/Features)
- What it is: A feature/score pair that publishes factor betas and aggregate risk signals for portfolios/holdings using a chosen risk model (e.g., market, size, value, momentum).
- Interface:
- Feature product: factor returns, exposures, residuals;
- Score product:
portfolio_id,as_of,factor_concentration,active_risk, and top drivers; - Use: front‑office PM tools, compliance pre‑trade checks, and CIO dashboards.
Why mesh‑native? Foundational products keep identity and time stable across systems; derived products carry semantics (return/contribution math, fee treatments) and compositions (joins, benchmark alignment), and the platform enforces governance (entitlements, lineage, disclosure text) as code—consistent with data‑mesh principles and Dehghani’s product‑as‑quantum framing.
Design moves that make this work everywhere
- Stabilize IDs & clocks in foundations (e.g.,
asset_id,instrument_id,portfolio_id, effective‑dating). - Name the product type up front (foundational vs. derived; if derived, which pattern: restricted, composite, semantic/metric, inferred/feature).
- Write the contract once: schemas, semantics, SLOs, quality rules, and version policy.
- Prefer zero‑/low‑copy interfaces (APIs/governed views) with row/column‑level security for restricted subsets.
- Automate governance at product boundaries (access, lineage, tests, disclosure text, retention) as part of a self‑serve platform and federated governance.
Why this framing scales
- The four principles give you the operating model (who owns what; how teams interoperate).
- The architectural‑quantum lens keeps products evolvable yet composable—no brittle central hub required.
- Domain examples (industrial telemetry and portfolio data) show the same four derived patterns, just with different nouns.