FabCon and SQLCon 2026 made the Microsoft Fabric and Azure Databricks story more concrete. The headline changes were not cosmetic. Microsoft moved zero-copy access to OneLake data from Azure Databricks into public preview, made Direct Lake in OneLake generally available, kept expanding Databricks-to-Fabric mirroring, and then pushed shortcut transformations into general availability in April. Put plainly, the platform story is moving away from “pick one stack forever” and toward “publish governed products that multiple engines can use.”
That matters because too many teams still call a dataset a product. Microsoft’s current guidance is more precise than that. A data product has defined shape, interfaces, maintenance expectations, and refresh cycles. It is processed for analytical use, and it should be discoverable, secure, interoperable, and valuable enough to serve downstream consumers without forcing them back into raw source complexity. The OneLake Catalog is now described in the Cloud Adoption Framework as a unified access surface for approved data products across Fabric and external processing platforms such as Databricks.
That is where Quantum Regression is a useful teaching device. I am not using it here as a product feature – or even going into the technology -, I am using it as a way to think about how data products enable modular expansions to the data estate. In this framing, a data product is an independent transformation: it accepts ingestions, applies logic, enforces quality and policy, and surfaces results. The file, table, semantic model, or API is only the current observable state of that transformation. The product is the operator, not the artifact. That is the mindset shift that helps make interoperability useful instead of merely fashionable.
Financial services teams have a familiar argument: “Are we a Databricks shop or a Fabric shop?” It sounds like a strategic question, but it usually hides the real problem—different parts of the business need different ways to use the same data, under tight controls, with clear auditability.
When Databricks and Microsoft Fabric interoperate at the data product level, the conversation shifts from which platform to how the data must be used: BI and semantic models, heavy Spark engineering, real-time analytics, governed sharing across boundaries, or advanced ML. The platform becomes a means, not the decision.
In this post I’ll lay out what “data product level interoperability” looks like in practice, why it enables responsible best-of-breed choices in regulated environments, and how it plays out in both directions: Databricks → Fabric and Fabric → Databricks.
Unity Catalog represents a paradigm shift in many ways, but one of the most underappreciated elements is its views. These views represent a massive change to the way we think about the medallion architecture and the lakehouse. In addition to being a critical part of the effort to reduce the number of copies of data that exist around an organization’s data estate, they also can form the building blocks of Data Mesh and Data Fabric architectures and can even be leveraged to create data products, which all become especially powerful as an organization increases its data strategy maturity and its analytics maturity. With two conceptually simple features, Databricks has turned the view from a useful part of the data engineering toolbox into an indispensable tool for every architect and even for citizen developers, especially in the modern day of generative AI.
Before diving into the Unity Catalog, it’s important for us to define our terms, especially those that are a part of the Medallion architecture. This approach organizes data processing in three layers: bronze, silver, and gold.
The bronze layer is where raw data is ingested from various sources. This is the first step in the data processing pipeline, where data is collected and stored in its raw form.
In the silver layer, the data is cleansed, transformed, and enriched. This is where the data is processed and prepared for analysis, with any errors or inconsistencies being corrected.
The gold layer contains business-level aggregates and metrics that are ready for decision-making processes. This is the final stage of the data processing pipeline, where the data is presented in a form that is easily understandable and actionable for business users.
This tiered approach helps in managing data lineage clearly and efficiently, making it easier for organizations to trace how data transforms across its lifecycle. The Medallion architecture is by far the most common architecture for Lakehouses, and we will be referencing it throughout this piece, but Unity Catalog can work with any of the many different reference architectures, notably Data Vault 2.0 or Lambda.
Building Blocks for a new Lakehouse
By combining views with UDFs and Unity Catalog’s permission model, we start to build a new, more fluid approach. Users (or in the best practice, group members) are then able to see, using customized permissions, exactly what they are allowed to see, but only one object must exist. Even more powerfully, the view executes as the view’s owner, meaning that the user does not need permissions to either the underlying data, or to the functions that are used to make the view work.
Using this principal, there are three building blocks that can be used to create highly dynamic tables for end users:
Row-Based Access Control – Users can be given access to only the rows that they need to do their jobs, limiting the scope of their access. This allows users to focus on what they need to do and prevents the accidental overexposure of data.
Column-Level Access Control – Specific columns can be shown or hidden based on a user’s permissions. This allows fine-grained access to data while respecting both privacy and security.
Dynamic Data Masking – When most people think of masking, they think of the kind of masking applied to credit card numbers, and that is definitely possible with Unity Catalog views, but there a whole universe of masking available to a developer in Databricks now. This dynamic masking can occur at a row or column level and can be used to restrict access to specific restricted information while still granting it to those with need to know. In this way, aggregations can be maintained without the need for complex rollup tables.
All these capabilities come together to provide a new, data governance integrated experience that allows for duplicate effort and data to be minimized across a data estate. With this end-users and citizen data scientists can be safely given access to data to enable them to use their domain-level expertise to find more possibilities. That brings us to an evolving paradigm in the data warehousing world.
How Views Change the Lakehouse
In a traditional lakehouse, data is copied into each layer as it is transformed. This represents an opportunity for inconsistencies and stale – or even bad – data to become a part of our final analysis set. This copying also means that the entire data warehouse must be reloaded from scratch when changes are made, reducing the nimbleness of the data warehouse and making it impossible to represent as highly consistent and repeatable code, rather than changeable and expensive data. The view changes this, but making it possible to reduce the number of copies of the data from three (or more) to just two – the data in the bronze layer that has not yet been transformed, and the clean enriched data in the silver layer. Gold simply becomes a view (or many views) on top of the silver layer.
This change drastically reduces the amount of work needed to create a secure lakehouse. Under previous models, it would often be necessary to create entirely different tables for each persona, and move data into them, creating significant delays and data duplication. This also introduced an opportunity for data to become out of sync, or for the wrong data to be copied into the wrong place. Increasing the risk of a lakehouse compared to a traditional data warehouse.
Enabling Data Products and Data Mesh
This makes Unity Catalog views an ideal way to represent data products and to build into a Data Mesh architecture. By creating a pattern that can be used to represent both complex, IT/Data driven data products as well as local, departmentally produced data products, the paradigm builds the foundation for a true data mesh architecture.
Non-IT organizations can build out and provide their own data products based on data that they have that is integral to a specific use case, and customize it to many different personas – and these personas do not have to be the same personas used by other data products – each data product can be totally a totally independent member of the organizations data estate.
Following on to that, though, is the ability for views (and these view-based data products) to build upon other views. This truly exemplifies the data mesh architecture, and the dynamic way all these data models interact can give a persona truly bespoke data model access. Not simply to the specific rows and records that they need, but customizing data display within those records, ensuring that data availability is maximized while data privacy and data security are not compromised.
Unity Catalog represents a significant shift in the way we think about data management and the Medallion architecture. By reducing the number of copies of data and enabling the use of views, the Unity Catalog improves data governance and reduces the risk of inconsistencies and stale data. Furthermore, the Unity Catalog enables the creation of data products and the implementation of a data mesh architecture, allowing for more dynamic and flexible data management. Overall, the Unity Catalog has the potential to revolutionize the way organizations manage their data, improving efficiency, security, and flexibility.
I wrote up some thoughts on how Databricks’ Unity Catalog Views change the way we should be thinking about lakehouses, and bout how they enable data mesh architectures on the Neudesic Innovation channel on Medium
