Kindly fill up the following to try out our sandbox experience. We will get back to you at the earliest.
A data fabric unifies access to data where it already sits. See the six layers, the architecture diagram, and when a data fabric is the wrong choice.
Updated on
.png)
A data fabric is an architecture in which a single metadata driven layer sits over every system that holds data and delivers governed, unified access to all of them, without first consolidating that data into one store. The data stays where it is. What gets centralized is the knowledge of what exists, what it means, who may see it and how to get it.
That is the whole idea, and the word doing the work in it is metadata. A data fabric reads the schemas, the lineage, the usage patterns and the quality results from every connected system, and then uses what it learns to automate the delivery: which sources answer a given request, whether to query them in place or serve a materialised copy, which permissions apply, and which pipeline to flag when something upstream changes.
The term is used loosely enough that it is worth having a test. A design deserves the name data fabric when it passes all three of the following.
A design that passes the first two and fails the third is common, and it is the version that gets an organization into trouble. Unified access without a unified policy model means you have made every silo reachable by more people without deciding who should reach it.
Traditional data management grew up around separate systems and separate teams, and the result is fragmentation: the same customer described four ways in four places, no single view of anything, and an analyst who spends the first day of any question working out which system to believe. Every attempt to fix this by consolidation runs into the same wall, which is that some systems are never moving. The regulated core banking platform, the manufacturing execution system, the acquired subsidiary running its own stack, the mainframe with two decades of history in it.
A data fabric is the response to that wall. Rather than continuing to argue for a migration that will not happen, it puts a layer over what exists and makes it usable together. It brings together data integration, data virtualization, metadata management and data governance into one design instead of four disconnected projects, and it gives the organization a single view of its data without requiring a single copy of it.
In practice, four conditions have to be present before a fabric earns its cost. Data lives in systems that cannot be consolidated for legal, contractual or practical reasons. More than one team needs the same data and each is building its own path to it. Access decisions are currently made per system, so nobody can answer who can see a given field. And there is enough change upstream that manually maintained integration keeps breaking. When fewer than three of those hold, the money is better spent elsewhere.
A data fabric architecture is usually drawn as six layers over the sources, with an active metadata layer reading from all of them and feeding automation back. The six are ingestion, processing, orchestration, discovery, data access and data management. The diagram below shows how they sit together and where the sources and consumers attach.
The connectors that reach each source and capture data from it, in batch, through change data capture, or as a continuous stream. This is where the old integration problem lives, and it is the layer most people mean when they say integration: bringing data from databases, applications, cloud platforms and connected devices into a coherent layer. The distinction that matters in a fabric is that capture and consolidation are separate decisions. Connecting a source does not commit you to copying it.
Transformation, standardization and joining, run either where the data already sits or in a shared engine. This layer is where distributed computing earns its place: processing is pushed out across many machines and, where possible, down to the source system, so volume grows without a single engine becoming the constraint. Scaling a fabric means adding processing nodes rather than rebuilding the design, which is the property that lets the same architecture serve a hundred sources as well as ten.
Scheduling, dependency management and retries. A report that runs on a table that is only half loaded is worse than a report that has not run, because it is wrong without announcing it. Orchestration is the layer that decides ordering and holds a downstream job until its inputs are complete.
The catalog, the business glossary and the semantic definitions. This is the layer that lets a person find a field, see what it means, see where it came from and know whether they are allowed to use it. Without it, unified access simply means a larger surface to search blindly, and the fabric becomes harder to use than the silos it replaced.
Delivery to whoever is asking, under one permission model. There are two delivery paths and a fabric uses both. Data virtualization creates a logical view that queries the source in place, so the reader gets current values with no copy and no movement. Materialisation writes a physical copy that answers faster and costs storage. The fabric chooses between them per request based on freshness requirements, source load and cost, which is one of the clearest examples of active metadata doing real work.
Governance, quality, lineage, retention and security, applied once across every connected source rather than reimplemented per system. This is the layer that turns unified access into safe unified access, and it is the layer most implementations postpone. A fabric built out of layers one to five reaches everything and protects nothing.
| Layer | What it owns | What breaks when it is missing |
|---|---|---|
| 1. Ingestion | Connectivity and capture from every source | Teams build point to point pipelines around the fabric, and the fabric becomes one more system to keep in sync. |
| 2. Processing | Transformation, standardization and joins | Every consumer transforms the same raw data its own way, so two dashboards disagree and both are defensible. |
| 3. Orchestration | Order, timing, dependencies and retries | Reports run on partially loaded inputs. The failure is silent and gets found by a business user rather than an alert. |
| 4. Discovery | Findability, meaning and ownership | People cannot tell which of six similarly named fields is the one they want, so they ask a data engineer, which is the bottleneck the fabric was bought to remove. |
| 5. Data access | Delivery, virtualised or materialised, under one permission model | Access is granted per system by whoever administers it, and nobody can answer who can see a given field. |
| 6. Data management | Policy, quality, lineage and retention | Unified access to ungoverned data. The blast radius of a bad field grows with every source you connect. |
The layers describe what a data fabric is made of. The principles describe how it has to be designed for the layers to hold together. Five are worth stating explicitly, because they are the decisions that get made badly.
These two get compared constantly and they are not the same kind of decision. A data fabric is a technology decision: build a central layer that unifies access, owned by a platform team. A data mesh is an operating model decision: give the domain teams that generate data the ownership of publishing it as a product, with federated governance holding the standards. One changes your architecture, the other changes who is accountable on Monday morning.
| Question | Data fabric | Data mesh |
|---|---|---|
| What kind of change is it | Technology and architecture | Operating model and ownership |
| Who owns the data | A central platform team owns the layer over everything | The domain team that generates the data owns it as a product |
| Where governance sits | Centralized in the fabric and pushed down to sources | Federated: global standards, local enforcement by each domain |
| What it needs to succeed | Metadata coverage and connectors for every source that matters | Domain teams with the engineering capacity to run their own data products |
| Main failure mode | A central layer nobody governs, so access spreads faster than control | Domains with no platform support, so each one reinvents pipelines and quality |
| Best when | Sources are many, distributed and cannot be consolidated | Domains are strong, well staffed, and central data engineering is the bottleneck |
They are not alternatives in any strict sense, and the more useful framing is that a fabric can be the platform a mesh runs on: the domains publish products, the fabric provides the discovery, access and policy machinery that makes those products findable and safe. If you are choosing between them, choose on the constraint you actually have. A central team drowning in requests from capable domain teams is a mesh problem. Fifty systems nobody can reach through one path is a fabric problem.
This page deliberately stops there, because the mesh deserves its own explanation rather than a paragraph. If you want the four principles the mesh is built on, start with what a data mesh is, and if you are weighing the two architectures against each other for a specific team, you can compare data mesh and data fabric side by side.
This is the comparison most vendor pages skip, and it is the one that decides budgets. A data warehouse and a data lakehouse are stores. They hold data that has been moved into them. A data fabric is a layer over stores, including those ones. The question is never fabric or warehouse. It is whether a second copy in a central store is the right way to answer a particular question.
| Question | Data fabric | Data warehouse or lakehouse |
|---|---|---|
| What it is | An architecture layered over the systems you already run | A store that data is copied into and modelled inside |
| Does data move | Only when materialisation is the better trade for that request | Yes, by design. Ingestion into the store is the first step |
| Where governance is applied | Once, across every connected source | Inside the store, for the data that reached it |
| What it is good at | Reaching systems that will never be migrated, and giving one governed path to all of them | Fast, repeatable analytics on modelled historical data at high concurrency |
| Typical weakness | Query performance on a virtualised path across slow or loaded sources | Anything not yet ingested is invisible, and the ingestion backlog is where the delay lives |
| Sensible combination | The fabric governs and serves everything, including the warehouse, and routes to it | The warehouse remains the fast path for modelled analytics inside the fabric |
The practical answer for most organizations is both. Keep the warehouse or lakehouse as the fast, modelled, high concurrency path for the analytics that justify a copy. Use the fabric to reach everything else, to govern the whole set under one policy model, and to decide per request which path serves it. A fabric that tries to replace the warehouse usually ends up rebuilding one.
Data virtualization is a technique. It creates a logical view over one or more sources and queries them in place at read time, so the consumer sees current data without a copy being made. It is genuinely useful and it is not an architecture.
A data fabric uses data virtualization as one of the two delivery methods inside its data access layer, and picks between virtualising and materialising per request. The other five layers are still needed. Virtualization on its own gives you a view; it does not give you the catalog that lets someone find the view, the lineage that explains where its columns came from, the orchestration that stops it reading a half loaded table, or the policy model that decides who may query it.
The short version, and it is worth remembering when a vendor uses the two terms interchangeably: every data fabric contains data virtualization, and a data virtualization tool on its own is not a data fabric. The gap between them is layers four and six.
This confusion is now common enough to be worth stating directly. Microsoft Fabric is the brand name of a specific Microsoft analytics platform. A data fabric is a vendor neutral architectural pattern that predates it. The two words are the same and the two things are different, and a search for one returns plenty of the other.
The distinction gets sharpest around AI readiness, because that is where teams are currently making the choice. A Fabric data agent is Microsoft's generally available feature for building a conversational question and answer experience over data. Microsoft's own documentation is specific about the shape of it: an agent connects to data governed in OneLake through lakehouses, warehouses, Power BI semantic models and KQL databases; it supports up to five data sources per agent; it is strictly read only and generates no write queries; it does not support unstructured files such as PDF or Word documents; it currently supports English only; and chat responses are capped at 25 rows and 25 columns because it is built for conversational insight rather than dataset retrieval.
Those are sensible boundaries for the job it does, and they define the job. A Fabric data agent makes data that is already inside Microsoft Fabric conversational. It is an excellent answer if your analytical estate is Microsoft Fabric.
A dedicated data context layer answers a different question, which is what an AI system is allowed to know and whether the answer it gives can be trusted. It works across every platform rather than one, so the Snowflake instance, the Postgres estate, the Databricks workspace and the Microsoft platform sit under one catalog. It carries column level lineage, so an answer can be traced back to the fields and systems that produced it. It holds quality results next to the metadata, so a stale or failing table is known to be stale before an agent quotes it. And it holds one set of access and sensitivity rules that apply regardless of which platform the request came through.
The practical difference is where the two things live. The agent lives inside one platform and answers questions about the data in it. The context layer sits above every platform and decides what any agent, in any platform, is permitted to see and how confident it should be in the answer. Most organizations that run Microsoft Fabric also run something else, which is why the two are usually complements rather than a choice: the agent handles conversation inside the platform, the context layer handles trust across all of them.
An identity data fabric applies the same architectural idea to identity data specifically. Most organizations hold identity attributes in several places at once: a directory service, one or more identity providers, a human resources system, a customer identity store and whatever the acquired business brought with it. An identity data fabric unifies those into one queryable identity view without forcing a migration into a single directory, so authorisation decisions and access reviews can be made against a complete picture of a person rather than a partial one.
The architecture is recognizably the same six layers, and the differences are in emphasis rather than in kind. Ingestion is dominated by directory and identity protocols. The discovery layer is concerned with reconciling the same person across systems that disagree about their attributes. And the data management layer carries far more weight, because identity data is almost always personal data and is therefore in scope for privacy regulation everywhere it is read.
It is worth knowing that this is a distinct product category from the analytics oriented data fabric that the rest of this page describes. If you are reading about identity data fabric architecture, you are usually looking at identity and access management vendors rather than data platform vendors, and the buying centre is the security team.
Five things, and they are worth separating from the marketing version because each one has a cost attached as well as a benefit.
On the second of those, governance is the point at which a fabric stops being an integration exercise. Centralized metadata management, data lineage tracking and access controls are what let an organization say who touched a field, where a number came from and who is allowed to see it, across every connected system rather than one at a time.
The pattern shows up wherever an organization holds the same subject in several systems it cannot merge. Five industries account for most real deployments.
The regulated cases deserve a note of their own, because they change what the data management layer has to produce. A financial institution supervised by OJK in Indonesia, APRA in Australia or MAS in Singapore, or an insurer under NAIC requirements in the United States, is asked for more than governing its data well. It has to demonstrate, on request, where a reported number came from and who could see the underlying record. That is a lineage and access evidence requirement, and it is the reason layer six cannot be the phase two of a fabric program in those industries.
Almost nothing written about data fabrics says this, so it is worth saying plainly. There are five situations where building one is the wrong move, and in each case there is a test you can apply before spending anything.
One further caution that applies even when a fabric is right. The build is a metadata project before it is an integration project, and metadata coverage is the usual point of failure. If the sources are poorly documented, the automation the design depends on has nothing to work with, and the fabric degrades into an expensive set of connectors.
Implementation follows the same sequence as any other integration program, with one difference that matters: the metadata work is not a later phase, it is the thing the rest depends on. Seven steps, in order.
Decube is not a data fabric and does not claim to be one. It covers layers four and six, which are the two that most fabric programs postpone and then rebuild after an incident: discovery, and data management.
On the discovery side, the data governance platform provides the catalog, the business glossary and column level lineage, so a field can be found, understood and traced back to the system that produced it across every connected source. On the data management side, automated checks on freshness, volume and distribution through the data observability platform mean a virtualised view is known to be stale before somebody quotes it in a board paper. Access and sensitivity rules are held once and applied consistently rather than per database.
Pricing is published, so an evaluation does not require a conversation first. The Starter plan is 175 USD per user per month from 21,000 USD a year with a minimum of ten users, and the Growth plan is 225 USD per user per month from 54,000 USD a year with a minimum of twenty users. If you want to see the catalog, lineage and quality views running against your own systems before deciding what your fabric layers four and six should look like, book a demo and bring the two sources you already know disagree.
A data fabric is worth building when the sources genuinely cannot be brought together and the cost of that separation is being paid every week in reconciliation, delay and arguments about which number is right. It is worth not building when the estate is small enough to fix directly, when the underlying data is wrong rather than merely scattered, or when nobody has been made accountable for the metadata that the whole design runs on. The architecture is the easy part. The decision about which of those situations you are actually in is the part worth getting right.
A data fabric is an architecture in which a single metadata driven layer sits over every system that holds data and delivers governed, unified access to all of them, without first consolidating that data into one store. The data stays where it is, and what gets centralized is the knowledge of what exists, what it means, who may see it and how to reach it. It is a design pattern rather than a product, so vendors sell components of it rather than the whole thing.
Data fabric means a layer of connective architecture over distributed data. The word fabric is meant literally: rather than moving everything into one store, the design weaves the existing systems together so they can be queried, governed and delivered as though they were one. The thread holding it together is active metadata, which records what each system contains and then drives the automation that serves requests from it.
The definition that separates a data fabric from ordinary integration has three parts. It connects sources without requiring them to be consolidated into a single platform. Its metadata is active, meaning it drives joins, caching, access decisions and alerting rather than merely describing what exists. And it applies one policy model for access, sensitivity, retention and quality across every connected source. A design that fails any of the three is something else, usually a data catalog or an integration platform.
Data fabric architecture is the six layer structure a data fabric is built from: ingestion, processing, orchestration, discovery, data access and data management, with an active metadata layer reading from all six and feeding automation back into them. Ingestion connects the sources, processing transforms, orchestration handles order and timing, discovery provides the catalog and meaning, data access delivers either a virtualised view or a materialised copy, and data management applies governance, quality and lineage once across everything.
A data fabric architecture diagram shows three bands. At the top are the source systems, which stay where they are: operational databases, warehouses and lakehouses, applications, object storage, event streams and on premise systems. In the middle are the six layers of the fabric, ingestion through data management, with the active metadata layer running across all of them. At the bottom are the consumers: business intelligence, machine learning, operational applications, AI agents and regulatory reporting. The diagram in this article is drawn that way.
Data virtualization is a technique that creates a logical view over one or more sources and queries them in place, so the reader sees current data without a copy being made. A data fabric is an architecture that uses data virtualization as one of two delivery methods inside its data access layer, choosing between virtualising and materialising per request. Virtualization gives you a view. It does not give you the catalog that lets someone find the view, the lineage explaining where its columns came from, the orchestration that stops it reading a half loaded table, or the policy model deciding who may query it.
An identity data fabric applies the same six layer architecture to identity data specifically, unifying attributes held across a directory service, one or more identity providers, a human resources system and any customer identity store into a single queryable view without forcing a migration into one directory. The emphasis differs: ingestion is dominated by directory and identity protocols, discovery is mostly about reconciling the same person across systems that disagree, and the data management layer carries more weight because identity data is almost always personal data. It is a distinct product category, usually sold by identity and access management vendors rather than data platform vendors.
Five principles decide whether the design holds together. Metadata is active, so it drives behavior rather than describing it. Data stays where it is by default, and movement is justified per case. One policy model is defined in the fabric and enforced down at the source, because a policy that only applies to traffic through the fabric can be bypassed by querying the source directly. The semantic layer is shared rather than rebuilt per consumer, so one certified definition exists per business metric. And the automation carries a measured target, such as how long a new source takes to become queryable, because unmeasured automation is not noticed when it stops working.
Data fabric design is the work of deciding, for a specific organization, which sources the fabric reaches, how much metadata coverage exists to automate with, where each request should be served from, and how policy is enforced at each source. The architecture is standard. The design is where it meets an estate: which systems will never be migrated, which queries are too heavy for a virtualised path and must be materialised, who owns the glossary, and which layer gets built first. In most cases the honest first answer is discovery and data management, because the delivery layers are worth little without them.
In seven steps. Map the systems, formats and access problems you actually have. Define the objectives and the use cases the fabric has to serve. Choose a solution on connector coverage for your real sources, metadata depth and security model. Plan the rollout in phases or as a pilot rather than as one program. Establish governance and security before widening access, which is the sequencing mistake that costs the most when it is made backwards. Involve the people who will use it, with training. Then monitor and iterate against a number you agreed in advance, such as how quickly a new source becomes queryable.
They solve different halves of the problem and are usually complements. A Microsoft Fabric data agent makes data that already sits inside Microsoft Fabric conversational: per Microsoft's documentation it connects to lakehouses, warehouses, Power BI semantic models and KQL databases in OneLake, supports up to five data sources per agent, is strictly read only, does not handle unstructured files, supports English only, and caps chat responses at 25 rows and 25 columns. A dedicated data context layer sits above every platform rather than inside one, and answers what an AI system is allowed to know and whether its answer can be trusted: one catalog across Snowflake, Databricks, Postgres and Microsoft alike, column level lineage so an answer traces back to the fields that produced it, quality results held next to the metadata so a stale table is known to be stale before an agent quotes it, and one set of access and sensitivity rules regardless of which platform the request arrives through. If your entire estate is Microsoft Fabric, the agent may be enough. If it is not, the context layer is what makes AI readiness true across all of it.
First step to AI Readiness
Table of Contents
Sneak peek from the data world.
Bring lineage, quality, governance, and AI together in one platform—so your teams and AI agents can act with confidence.
_For%20light%20backgrounds.svg)
.svg){kind=small}
.webp)