Data Mesh, Data Products and Platform Thinking

Senior and staff interviews increasingly include “how would you organise data for a company with N domains?” This is about architecture and people.

1. Centralised vs federated

flowchart TB
    subgraph Central["Centralised data team"]
        S1[Sales] --> CT[Central DE team<br/>bottleneck] --> DW[(One warehouse)]
        S2[Marketing] --> CT
        S3[Logistics] --> CT
    end
    subgraph Mesh["Data mesh"]
        D1[Sales domain team<br/>owns sales data products] --> P[(Self-serve platform<br/>storage, compute, catalog, CI)]
        D2[Marketing domain] --> P
        D3[Logistics domain] --> P
        GOV[Federated governance<br/>global standards] -.-> D1
        GOV -.-> D2
        GOV -.-> D3
    end

2. The four data mesh principles (Zhamak Dehghani)

1. Domain ownership: the teams closest to the data own it end to end, including quality and SLAs.
2. Data as a product: datasets have owners, documentation, SLAs, versioning, discoverability. Consumers are customers.
3. Self-serve data platform: a platform team provides paved roads (ingestion templates, CI/CD, catalog, compute, monitoring) so domains don’t each reinvent infrastructure.
4. Federated computational governance: global rules (PII handling, naming, interoperability, quality minimums) agreed centrally and enforced automatically by the platform.

3. What a “data product” looks like

4. Multi-domain lakehouse reference layout

flowchart LR
    subgraph Platform["Platform team"]
        ING[Ingestion frameworks<br/>CDC, AutoLoader templates]
        CI[CI/CD + dbt project template<br/>shared macros]
        GOVP[Catalog, tags, policies,<br/>PII scanning]
        MON[Monitoring, cost dashboards]
    end
    subgraph Domains["Domain catalogs"]
        SALES["sales catalog<br/>bronze / silver / gold"]
        FIN["finance catalog"]
        MFG["manufacturing catalog"]
    end
    Platform --> Domains
    SALES -->|"shared, versioned<br/>data products"| FIN
    MFG --> FIN

Practical patterns:

• One catalog per domain (per environment), schemas per layer; cross-domain consumption only via published gold data products (no reaching into another domain’s silver).
• Shared macro/library package (masking, surrogate keys, audit columns, DQ checks) versioned by the platform team so 8 domains don’t write 8 versions of the same logic.
• Templates (cookiecutter / asset bundles) for new pipelines: logging, alerting, tests, permissions pre-wired.
• Conformed dimensions (customer, product, calendar) owned by one domain and consumed by all.

5. When NOT to do data mesh

• Small company (< ~5 data engineers), few domains: centralised is faster and cheaper.
• Domains lack data skills or headcount: ownership on paper only.
• No platform investment: “mesh” becomes a mess of duplicated pipelines.

A common middle ground is hub-and-spoke: central platform + central core models (conformed dims), domain teams own their marts.

6. Cost management (FinOps for data)

Senior phrasing: “I’d tag every job and warehouse with domain and owner, publish a weekly cost dashboard per domain, and target the top 10 most expensive jobs first. It’s usually full refreshes that should be incremental, or tables with small-file problems.”

7. Interview questions