The Workspace Is Not the Product: Fabric Objects as Code and the Rise of Lakehouse as Code

A Microsoft Fabric deployment can finish successfully and still leave you with an empty Lakehouse.

The notebook arrives. The pipeline arrives. The Lakehouse item appears in the target workspace. References are rebound to the correct environment. Every deployment task is green. Then the team opens the Lakehouse and discovers that the schemas, Delta tables, Spark views, and materialized lake views expected by the solution are not there.

That outcome is not a contradiction. It exposes an architectural boundary that Fabric teams need to make explicit: deploying a Fabric item is not the same as constructing the state inside that item.

The distinction matters in any enterprise. It matters more in financial services, where a deployment may support card authorization controls, wealth positions, settlement operations, lending decisions, or insurance exposure reporting. A release that creates the application shell without the required internal structures has not delivered a usable data product, however successful the pipeline log may look.

This article updates the idea of Fabric objects as code for the platform Microsoft is shipping in 2026. Fabric now has a much more coherent delivery stack: Terraform for stable platform resources, Git-backed item definitions, Variable Libraries for configuration, fabric-cicd and Fabric CLI for scripted deployment, and materialized lake views for declarative Lakehouse transformations. The result is not one universal deployment tool. It is a layered architecture in which each form of state is managed by the mechanism suited to it.

Fabric Objects as Code is Now a Platform Architecture

The original case for Fabric objects as code was mostly defensive. Teams needed a way to stop production workspaces from becoming collections of undocumented clicks. Git integration helped, deployment pipelines helped, and notebooks could fill some gaps. The pieces existed, but the operating model often had to be assembled by each implementation team.

Microsoft’s current Fabric CI/CD overview is much clearer. It brings together Git integration, deployment pipelines, Fabric REST APIs, Variable Libraries, Fabric CLI, Terraform, and fabric-cicd as parts of one delivery platform. Microsoft’s new end-to-end automation tutorial goes further by separating a relatively stable control plane from a more frequently changing data plane: provision workspaces and related platform resources with Terraform, then move source-controlled Fabric items with Git and fabric-cicd.

That is a useful model, but a Lakehouse adds one more boundary. I treat Fabric delivery as three related forms of state:

  • Control-plane state includes capacities, workspaces, domains, connections, role assignments, tenant configuration, and Git wiring. Terraform, REST APIs, and Fabric CLI are appropriate here.
  • Workspace-item state includes notebooks, pipelines, Lakehouse metadata, semantic models, reports, and Variable Libraries. Git, deployment pipelines, fabric-cicd, and Fabric CLI move these definitions between environments.
  • Internal data-structure state includes schemas, Delta tables, Spark views, materialized lake views, table properties, and structural tests. Native idempotent code reconciles this state inside the deployed item.

This third plane is not a competing CI/CD product. It is the part of the release that makes the deployed data container internally usable.

A complete Fabric release manages platform resources, workspace items, and the structures inside data items as separate forms of state.

Git Versions the Lakehouse Item, not Everything Inside it

Fabric’s Lakehouse lifecycle support has improved substantially. Lakehouse metadata and OneLake shortcut definitions can now participate in Git integration and deployment pipelines, and internal shortcut targets can be remapped across stages. That is meaningful progress because shortcuts are architectural objects, not merely data files.

The boundary remains important. Microsoft’s Lakehouse Git and deployment documentation states that Delta and non-Delta tables, Spark views, and folders in the Files section are not tracked in Git. It also states that, absent a dependency mapping, deployment creates a new empty Lakehouse in the target workspace and remaps dependent notebooks and Spark Job Definitions to it.

That behavior is sensible. A source-control update should not overwrite terabytes of production data. It does mean that a repository containing a Lakehouse definition is not, by itself, a complete declaration of the Lakehouse’s internal structure.

This is the job of the idempotent reconciler notebook.

The reconciler is sometimes misunderstood as an ingestion notebook or a scripted substitute for fabric-cicd. It should be neither. fabric-cicd publishes Fabric item definitions. The reconciler notebook is one of those published items, and its responsibility begins after the target Lakehouse exists. It inspects the target and brings its internal structures into alignment with the expected version.

A well-designed reconciler can safely run after every deployment. It creates missing schemas, creates tables that do not exist, applies explicitly approved schema migrations, establishes or replaces views and materialized lake views, verifies required table properties, and stops when it detects an unsafe difference. It does not casually drop production tables to make the deployment log green.

That last point is worth emphasizing. Idempotency means repeated execution converges on the intended result. It does notmean replacing every object every time. CREATE OR REPLACE is appropriate for a view or materialized lake view whose definition is the product. For a persistent Delta table containing business history, the safer pattern is usually CREATE TABLE IF NOT EXISTS, followed by deliberate ALTER TABLE migrations and an explicit failure for destructive drift.

A simplified card-processing example looks like this:

CREATE SCHEMA IF NOT EXISTS ingest;
CREATE SCHEMA IF NOT EXISTS transform;
CREATE SCHEMA IF NOT EXISTS surface;
CREATE TABLE IF NOT EXISTS ingest.card_authorization_event (
authorization_id STRING,
account_token STRING,
merchant_id STRING,
authorization_amount DECIMAL(18, 2),
authorization_timestamp TIMESTAMP
) USING DELTA;
ALTER TABLE ingest.card_authorization_event
SET TBLPROPERTIES (delta.enableChangeDataFeed = true);
CREATE OR REPLACE MATERIALIZED LAKE VIEW transform.valid_authorizations
(
CONSTRAINT valid_amount
CHECK (authorization_amount >= 0) ON MISMATCH FAIL
)
TBLPROPERTIES ("delta.enableChangeDataFeed"="true")
AS
SELECT
authorization_id,
account_token,
merchant_id,
authorization_amount,
authorization_timestamp
FROM ingest.card_authorization_event;
CREATE OR REPLACE MATERIALIZED LAKE VIEW surface.daily_network_exposure
AS
SELECT
CAST(authorization_timestamp AS DATE) AS business_date,
SUM(authorization_amount) AS authorized_amount
FROM transform.valid_authorizations
GROUP BY CAST(authorization_timestamp AS DATE);

Production code would include migration history, ownership metadata, comments, validation queries, and more careful handling of table properties. The important point is structural: the same source-controlled notebook can be executed in development, test, and production, and each environment converges on the same expected object model without rebuilding the data indiscriminately.

Materialized Lake Views Make Lakehouse as Code Credible

Lakehouse as code used to be a useful aspiration with an awkward implementation. Teams could source-control notebooks that happened to create tables, but the transformation graph, refresh order, data-quality behavior, and operational state were largely custom concerns. The code described a procedure more often than it declared a durable Lakehouse model.

Materialized lake views change that equation.

Materialized lake views reached general availability in March 2026. They allow teams to define persisted Lakehouse transformations using Spark SQL; PySpark authoring is also available in preview. The current MLV overview describes results persisted as Delta tables, dependency-aware refresh, built-in quality constraints, monitoring, and lineage across Lakehouses. Spark SQL definitions can use CREATE OR REPLACE, partitions, comments, table properties, and CHECKconstraints with either DROP or FAIL behavior.

The operational story has also matured. Fabric’s optimal refresh can choose to skip a refresh, process incrementally, or perform a full rebuild based on source changes and supported query patterns. Incremental processing requires Delta sources and Change Data Feed, and PySpark-authored MLVs currently use full refresh. Those constraints matter, but they are visible engineering choices rather than hidden orchestration code.

In July, Microsoft added generally available MLV analytics and insights for refresh trends, failures, and recommendations. Event-driven refresh and a Data Factory refresh activity are available in preview. Public APIs can create schedules, run on-demand refreshes, monitor jobs, and define a subset of the lineage for independent execution.

Taken together, those capabilities give Lakehouse as code three characteristics it previously lacked:

The structure is declarative. The repository contains definitions of the curated objects, not only procedural notebooks that happen to produce them.

The dependency graph is operational. Fabric derives refresh order from the definitions rather than requiring a team to rebuild the graph as a pipeline of notebook activities.

The result is observable. Refresh history, quality violations, execution metrics, lineage, and recommendations are part of the platform instead of a parallel monitoring project.

Source-controlled definitions establish the internal Lakehouse structure, while Fabric manages materialization, dependency order, refresh, lineage, and operational insight.

For a card processor, that may mean landing authorization, clearing, chargeback, and reference data into ingest.*; using notebooks and MLVs to create conformed, quality-controlled structures in transform.*; and publishing settlement positions, exception queues, and daily network exposure through surface.*. Risk, finance, operations, compliance, Power BI, and data agents consume the same governed surfaces instead of implementing their own copies of the transformation logic.

Not every transformation belongs in an MLV. API calls, machine-learning inference, complex Python logic, and subsecond streaming belong elsewhere. The architecture becomes stronger when MLVs are used for the relational transformations they fit, while notebooks, Spark Job Definitions, pipelines, and Real-Time Intelligence handle the work they fit better.

fabric-cicd and Fabric CLI complete the item-delivery loop

Native idempotent code solves internal construction. It does not remove the need to deploy Fabric items correctly.

fabric-cicd remains important because it reads source-controlled item definitions and publishes them with Fabric’s create-or-update semantics. It handles dependencies and supports environment-specific replacement through parameter files. Microsoft now describes it as the most widely adopted Fabric deployment tool, and the end-to-end automation tutorial uses it as the data-plane deployment mechanism.

Fabric CLI is converging with the same model. Version 1.5 introduced a deploy command integrated with fabric-cicd. The Fabric CLI 1.7 release, published August 18, added environment-definition support, bulk export, and an opt-in experimental --bulk_publish deployment mode. Variable Libraries also gained broader CLI and API support earlier in 2026.

This convergence is more important than the individual commands. Teams no longer need to choose between “native Fabric” and “real engineering.” Git integration supplies reviewable definitions. Variable Libraries carry stage-specific values. fabric-cicd and Fabric CLI provide scriptable delivery. Terraform provisions stable resources. The reconciler notebook constructs the internal state. MLVs express and operate the Lakehouse transformation graph.

Each tool has a bounded responsibility, which is exactly what makes the architecture repeatable.

A release should converge, not merely copy

A practical Fabric release can now follow one consistent sequence:

  1. Provision or update the control plane. Terraform manages capacities, workspaces, connections, role assignments, and Git wiring when those relatively stable resources change.
  2. Commit the complete definition. The repository contains Fabric item definitions, parameter files, Variable Library references, the reconciler notebook, MLV definitions, structural tests, and migration logic.
  3. Validate the pull request. Automated checks inspect source formats, naming standards, dependencies, prohibited hard-coded identifiers, MLV SQL, and the safety of proposed schema changes.
  4. Publish workspace items. fabric-cicd or Fabric CLI creates or updates the target items and applies environment-specific bindings.
  5. Reconcile the internal structures. The deployed notebook creates missing objects, applies safe migrations, replaces declarative views and MLVs, and fails on unapproved destructive change.
  6. Refresh and verify. The release triggers the required MLV lineage or execution definition, runs structural and data-quality smoke tests, and records evidence for the release.
The same definitions move through development, test, and production; environment values change, while the expected internal structure remains constant.

This sequence also clarifies what production promotion means. The team is not copying a development workspace and hoping every reference survives. It is applying a versioned definition to a target, reconciling the target’s current state, and proving that the expected result exists.

That is a much better fit for financial services. A wealth-management position surface may retain years of transaction history while its materialized business views evolve. A lending platform may add a new decision attribute without rebuilding every loan record. A property and casualty insurer may change catastrophe-exposure aggregations while preserving the underlying policy and claims history. The object definitions move; the governed data remains under explicit control.

The standard should be reproducibility, not portal discipline

Fabric will continue to have a productive user interface, and teams should use it. The mistake is treating the workspace as the only durable description of what has been built.

The workspace is a runtime environment. The product is the set of definitions, configuration, migrations, tests, and operational behaviors that can reproduce the intended state in another environment.

The updated Fabric platform makes that model practical. Terraform can establish the platform. Git can version supported items. fabric-cicd and Fabric CLI can publish those definitions. Variable Libraries can separate code from stage-specific configuration. Idempotent notebooks can maintain the internal structure that Lakehouse Git intentionally does not carry. Materialized lake views can turn much of the Lakehouse transformation layer into declarative, dependency-aware, observable code.

The next useful test is simple: deploy a Lakehouse into a clean workspace and ask whether the release can build every required schema, table, view, and materialized lake view without a person repairing the result in the portal. If it cannot, the workspace is still carrying knowledge that belongs in the product definition.

Fabric objects as code is no longer just a source-control preference. It is the operating model that turns a collection of Fabric items into a platform you can promote, verify, and trust.

Unknown's avatar

Author: Jason Miles

A solution-focused developer, engineer, and data specialist focusing on diverse industries. He has led data products and citizen data initiatives for almost twenty years and is an expert in enabling organizations to turn data into insight, and then into action. He holds MS in Analytics from Texas A&M, DAMA CDMP Master, and INFORMS CAP-Expert credentials.

Leave a Reply

Discover more from EduDataSci - Educating the world about data and leadership

Subscribe now to keep reading and get access to the full archive.

Continue reading