Skip to content

DataMesh.md

Latest commit

 

History

History
292 lines (262 loc) · 27.2 KB

DataMesh.md

File metadata and controls

292 lines (262 loc) · 27.2 KB
Raw
Name Data Mesh
Category Documentation
Description Turn scattered data into addressable, combinable data products with clear ownership and governance
Icon /static/DocContent/DataMesh/icon.svg

The Data Problem

Every enterprise has the same story. Underwriting keeps the truth in Oracle. Risk keeps a different truth in SQL Server. Finance keeps yet another in SAP. When a question crosses those boundaries, someone has to spend a week reconciling three spreadsheets and hoping the columns line up.

Underwriting Oracle DB Excel SharePoint Legacy App Risk SQL Server Python R Studio Excel Claims Claims DB Excel Doc Store Imaging Finance SAP Power BI Postgres Excel Reserving Access DB MongoDB REST APIs Excel Each domain maintains its own version of the truth. Reconciliation is a full-time job.

AI Amplifies the Chaos

One person with AI agents can now produce what used to take a team. More output means more data — and more data without structure means more chaos.

1 Person Research Agent Analysis Agent Reporting Agent Execution Agent Market scans Risk models Board decks System updates 💥 exponential output

The flood of AI-generated artefacts — models, scans, reports, system updates — lands in the same fragmented landscape. Without a consistent way to address, own, and combine that output, agents just add noise to noise.

The Missing Piece: Combinability

Take this contract here, combine it with that portfolio there, overlay this exposure data, and show me the result.

That sentence is easy to say and brutally hard to execute across isolated silos. The missing ingredient is not more storage — it is combinability: a shared addressing scheme, clear ownership, and agreed contracts between producers and consumers.

Underwriting Risk Claims Finance Reserving Data Mesh addressable & governed Knowledge Capital Profit

Data Products: The Building Block

A data product is a self-contained unit of data with clear boundaries. It has a unique address in the mesh, a named owner who is accountable for its quality, a typed schema that acts as a contract with consumers, and explicit service-level commitments about freshness and availability.

Data Product @org/product-name Owner accountable person Schema typed contracts Service Levels quality & freshness Access Control read & write perms Change Cycles versioned evolution Address unique mesh path

Each attribute is non-negotiable. Without an address, the product cannot be referenced. Without an owner, quality erodes. Without a schema, consumers cannot trust the data. Without service levels, pipelines break silently.

From Silos to a Mesh

Data products become nodes in a graph. Arrows show data flowing from producers to consumers, with the agreed freshness commitment labelled on each connection — real-time, T+1, weekly, monthly, quarterly, or annual. The mesh makes those commitments visible and auditable instead of buried in hand-off emails.

Underwriting Risk Finance Reserving Submission Offering Policy Binding Exposure Cat Model Accumulation Loss Scenario Premium Claims Paid P&L Combined Ratio Triangle IBNR Case Reserve Dev Factor T+1 real-time T+1 monthly weekly quarterly annual

How MeshWeaver Implements This

MeshWeaver provides a complete set of building blocks for Data Mesh, each documented in its own guide:

Capability What it covers
Node Types Design, compile, NuGet-reference, and test node types end to end
Addressable Paths Every product gets a permanent, unique address in the mesh
Addressability of Objects Why one addressable namespace lets an agent go from a plain-English ask to a precise, typed action
Query Language GitHub-style search syntax to discover and filter across products
CRUD Operations Type-safe create, read, update, and delete for any product
Node Operations Export, import, copy, and move node subtrees
Data Modeling C# records as the schema contract between producers and consumers
Data Cubes Dimensions, FX conversion, and live slice-and-dice pivot tables and charts
Satellite Entities Comments, approvals, access, and audit trails attached to any node
Interactive Markdown Embed live data and charts directly inside documentation
NuGet Packages Reference any NuGet package from interactive markdown with #r "nuget:..."
Collaborative Editing Real-time co-editing with track changes
Data Configuration Wire data sources and hub-to-hub synchronization

Start Building

Follow this sequence to go from raw data to a governed, combinable data product:

  1. Model — Define your types and schema: Data Modeling + Node Type Configuration
  2. Address — Give every product a permanent home and reach every facet by path: Unified Path + Addressability of Objects
  3. Operate — Wire up reads, writes, and sync: CRUD + Data Configuration
  4. Govern — Attach ownership, access, and audit trails: Satellite Entities
  5. Consume — Surface live data in docs and dashboards: Interactive Markdown