A Data Product Is an Engine, Not a Table: FabCon 2026, Databricks, Fabric, and the Case for Interoperability

FabCon and SQLCon 2026 made the Microsoft Fabric and Azure Databricks story more concrete. The headline changes were not cosmetic. Microsoft moved zero-copy access to OneLake data from Azure Databricks into public preview, made Direct Lake in OneLake generally available, kept expanding Databricks-to-Fabric mirroring, and then pushed shortcut transformations into general availability in April. Put plainly, the platform story is moving away from “pick one stack forever” and toward “publish governed products that multiple engines can use.”

That matters because too many teams still call a dataset a product. Microsoft’s current guidance is more precise than that. A data product has defined shape, interfaces, maintenance expectations, and refresh cycles. It is processed for analytical use, and it should be discoverable, secure, interoperable, and valuable enough to serve downstream consumers without forcing them back into raw source complexity. The OneLake Catalog is now described in the Cloud Adoption Framework as a unified access surface for approved data products across Fabric and external processing platforms such as Databricks.

That is where Quantum Regression is a useful teaching device. I am not using it here as a product feature – or even going into the technology -, I am using it as a way to think about how data products enable modular expansions to the data estate. In this framing, a data product is an independent transformation: it accepts ingestions, applies logic, enforces quality and policy, and surfaces results. The file, table, semantic model, or API is only the current observable state of that transformation. The product is the operator, not the artifact. That is the mindset shift that helps make interoperability useful instead of merely fashionable.

The Quantum Regression view of a data product

Imagine a data product that requires a Quantum Regression step, run on a quantum computer. Inputs arrive. A bounded transformation operates. Outputs are surfaced from the quantum computer and are stored. If the storage engine, notebook runtime, or reporting surface changes, the business object should not have to change with it. The contract should remain stable even when the implementation evolves. Current data best practice points in the same direction: ingest data only when it supports a business outcome, certify gold datasets as business data products, and standardize how those products are published and governed.

A finance product that ingests ERP transactions, rebate files, price lists, and customer hierarchies works the exact same way. The important thing is not the bronze table by itself, and it is not even the polished gold table by itself. The important thing is the governed process that reconciles those ingestions, resolves lateness, applies business rules, and publishes a stable result with a clear contract. You can have literally any middle layer you want – on any platform that makes sense for that layer.

Once you see the product this way, an important misconception starts to disappear. A data product is not “the customer table in Fabric” or “the sales model in Databricks.” It is the transformation that creates a trusted customer view or sales view, plus the interface, ownership, lineage, and refresh promise around it. The table is a surface. The semantic model is a surface. The product is the repeatable, independent transformation underneath them – and the bounded expectations that define the outcome.

What FabCon changed in practical terms

The most consequential FabCon announcement for this conversation was OneLake catalog federation in Azure Databricks, now in public preview. In practical terms, Unity Catalog can query Fabric Lakehouse and Warehouse data in OneLake without copying it. Microsoft’s documentation is explicit that this access is read-only today, supports Lakehouse and Warehouse items, and currently does not support views, materialized views, or complex types such as arrays, maps, and structs. That is a meaningful step because it turns OneLake into more than a Fabric-only storage boundary. It becomes a governed publication layer that Databricks compute can read directly.

The reverse path is also mature enough to matter. Fabric can mirror Azure Databricks Unity Catalog so Fabric workloads can read Databricks-managed data without moving or replicating the underlying data. Fabric mirrors the catalog structure and accesses the data through shortcuts. It also creates a read-only SQL analytics endpoint and supports Power BI reporting in Direct Lake mode against the mirrored Databricks item. For enterprises that need tighter network control, mirroring Databricks catalogs from workspaces behind private endpoints is now generally available.

Just as important, Microsoft’s Cloud Adoption Framework now says the quiet part out loud: keep Databricks where expertise or scale already exists, but require outputs to land in OneLake or connect through OneLake shortcuts so governance, security, and discovery remain centralized. The same guidance recommends surfacing final Databricks gold outputs in OneLake, using shortcuts when remote access is sufficient and mirroring when local governance or consumption patterns need it. That is not just compatibility guidance. It is a design pattern for how interoperable data products should be published.

FabCon also strengthened the argument that a product is a transformation rather than a file drop. Shortcut transformations reached general availability in April 2026. They can take structured files referenced by a OneLake shortcut and automatically convert them into managed Delta tables that stay in sync, with inherited lineage, permissions, and Purview policies. Microsoft also says Fabric polls for changes every two minutes. That is a small but telling shift: the platform is increasingly treating ingestion-plus-transformation-plus-surface as one managed motion rather than as a stack of disconnected plumbing.

The publication layer is stronger on the consumption side too. Direct Lake in OneLake is now generally available, which means semantic models can read directly from OneLake with stronger alignment to OneLake security and without the old refresh-heavy posture teams often associate with BI serving layers. FabCon also highlighted OneLake Table APIs as generally available, using Apache Iceberg REST Catalog and Delta Lake APIs for programmatic table management. Those are not side notes. They reinforce the idea that the product boundary should sit at a governed interface, not at a single engine.

Where the edges still are

Interop is better, but it is not magic. Mirrored Azure Databricks still has real limitations. It does not currently support views, materialized views, streaming tables, Delta Sharing tables, or tables with RLS or column masking policies. Catalog federation from Databricks into OneLake is still read-only. That means teams should resist the temptation to describe interoperability as “everything works everywhere now.” It does not. What works is the emerging pattern of stable publication contracts over specialized engines.

Security tells the same story. OneLake security will be generally available in the coming weeks, but the public documentation still lists OneLake security and OneLake data access roles as preview as of the current docs. That does not invalidate the direction of travel, but it does mean teams should distinguish between conference-stage momentum and current operational status when they make risk decisions.

FabCon also introduced extended capabilities in mirroring, including Delta change data feed and mirroring views. That is useful for reducing full refreshes and for preserving richer change semantics. But there is an important nuance: mirrored views are currently supported only for Snowflake in preview, not for mirrored Azure Databricks. Again, the lesson is the same. Product contracts must sit above platform-specific mechanics, because those mechanics still vary by source.

Why this lowers project risk

This is where the Quantum Regression analogy earns its keep. If the product is the transformation, then the compute plane can move without rewriting the business meaning. Databricks can own high-scale ingestion, machine learning, or heavy reconciliation. Fabric can own publication, discoverability, semantic serving, and governed reuse through OneLake, SQL analytics endpoints, and Direct Lake. A completely external system – our Quantum Computer – can own something ridiculously specific and complex. The consumer binds to the product contract, not to the implementation accident of where the code happened to run.

That is a much safer way to run a program. It narrows migration scope. It lowers rework when platform capabilities change. It makes coexistence a first-class design decision instead of a temporary compromise. Most importantly, it reduces the familiar risk pattern where teams spend months debating whether Fabric should replace Databricks or Databricks should stay the “real” platform, while the business still lacks a trustworthy, documented, reusable product. Microsoft’s current architecture guidance is increasingly explicit that the right move is not platform absolutism. It is architectural clarity about ingestion, transformation, publication, and governance boundaries.

A well-designed interoperable data product, then, has a simple profile. It has clear inputs. It has an independent transformation. It has quality rules, lineage, security posture, freshness expectations, and a defined publication surface. Whether that surface is mirrored into Fabric, queried from Databricks, served through Direct Lake, or exposed through cataloged tables is an implementation choice. The risk goes down when that choice is reversible.

Conclusion

FabCon 2026 did not just add features. It clarified the architecture. Fabric and Databricks interoperability is no longer best understood as a migration bridge. It is better understood as a publication-and-consumption model built around governed products in OneLake and specialized engines on either side.

That is why the real lesson is not about catalogs, shortcuts, or federated reads by themselves. It is about product design. A data product is not the dataset you happen to hand someone. It is the independent transformation that accepts ingestions, applies governed logic, and surfaces trusted results through stable interfaces.

When teams fund and govern that transformation as the product, they de-risk their projects. They stop tying business meaning to a single runtime. They gain room to evolve platforms without breaking consumers. And they finally get closer to the thing data programs usually say they want: a trusted product, not just another table.

Unknown's avatar

Author: Jason Miles

A solution-focused developer, engineer, and data specialist focusing on diverse industries. He has led data products and citizen data initiatives for almost twenty years and is an expert in enabling organizations to turn data into insight, and then into action. He holds MS in Analytics from Texas A&M, DAMA CDMP Master, and INFORMS CAP-Expert credentials.

Discover more from EduDataSci - Educating the world about data and leadership

Subscribe now to keep reading and get access to the full archive.

Continue reading