Build data products that survive production.
Date: October 19, 2026
A Fabric deployment can finish successfully and still leave the business with the wrong architecture. The tables are present, the dashboard opens, and the agent answers questions. Yet nobody can explain which data developers should use, whether a report consumer can bypass its security, or what happens when a source changes during a release.
Those are not missing features. They are missing decisions.
My recommendation is to design Fabric around maintained data products, with explicit arrival boundaries, transformation responsibilities, published interfaces, and operating controls. Choose the engine after deciding what the product promises. Use managed capabilities before commissioning another framework, but keep custom engineering where it earns its operating cost.
This page brings the patterns developed across EduDataSci together with current Microsoft guidance. It explains when I would use each pattern, what I would avoid, and how the choices work in banking, wealth management, lending, insurance, reinsurance, and card processing. The examples are illustrative designs, not claims about named customer implementations.
Landing zones and domains: establish ownership before construction
A landing zone is the enterprise foundation around Fabric, not another name for a lakehouse. The EduDataSci reference architecture separates Fabric’s tenant, capacity, and workspace concerns from the identity, networking, secrets, monitoring, and storage services around it.
Use this pattern whenever a workload is production-bound. A platform team should establish approved identities, network routes, capacity ownership, provisioning, release paths, and recovery expectations. Domain teams should own the products built within those boundaries. A banking domain should describe a durable responsibility such as servicing or credit risk, rather than a project scheduled to disappear next quarter.
Do not confuse governance with keeping workspace counts low. As Workspace Sprawl Isn’t Your Fabric Problem argues, stale ownership is more dangerous than another well-managed boundary. Separate workspaces when lifecycle, access, or operational responsibility genuinely differ; retire them when that purpose ends.
For a life insurer, policy administration, actuarial analysis, and distribution analytics may consume related data while requiring different release and access arrangements. That is a reason to define interfaces, not to force everyone into one workspace.
Landing Lakehouses: separate arrival from permission to use
The Landing Lakehouse pattern makes a distinction that rapid ingestion can conceal: source fidelity, development safety, and repeatable testing are different requirements.
A protected landing receives or references source-aligned data. It then exposes purpose-built surfaces: approved production data, minimized or tokenized development data, representative subsets, and stable test baselines. These are architectural contracts we implement, not automatically secured outcomes of naming schemas.
Use this pattern when several environments or teams consume sensitive operational data. Wealth-management developers need realistic relationships between households, accounts, advisors, and transactions. They usually do not need client names or tax identifiers. Tokenization alone should not be described as guaranteed anonymization; the remaining attributes and reidentification risk still need review.
For lending tests, maintain a named, deliberately refreshed “Known Copy” containing expected payoff, reversal, delinquency, and recovery cases. A test against changing production inputs is not reproducible merely because the code is versioned. Keep the baseline stable until an owner approves a new one.
My default is to keep the source-facing operation source-owned, publish approved surfaces outward, and make each environment’s selected surface explicit in configuration.
Ingest–transform–surface: publish the contract, not the workshop
Medallion architecture describes progressive refinement. It does not, by itself, tell a consumer which object is safe to depend on. The ingest–transform–surface pattern adds that missing distinction.
The ingest boundary preserves the declared source contract. Internal transformations implement business logic and quality controls. The surface publishes supported tables, semantic models, events, or application interfaces. Consumers should not depend on an intermediate join because it was convenient to discover.
Use this pattern for products with more than one consumer or an expected maintenance life. A card-settlement product should state its grain, reconciliation definition, currency treatment, freshness, ownership, and exception semantics. Its internal implementation can change without silently redefining “settled.”
Publish breaking changes through a new interface version and an explicit migration period. An additive column is not necessarily a breaking change; changing the meaning of an existing measure often is. My preference is a small, intentional surface over a comprehensive exposure of the engineering workspace.
Engine selection: choose the publication contract first
The question is not whether Lakehouse or Warehouse is more modern. It is who must build, operate, and change the product. Choose the Publication Contract, Not the Fabric Icon makes that an ownership decision rather than a branding exercise.
Microsoft’s data-store decision guide differentiates SQL database for transactional applications, Cosmos DB for document-oriented workloads, Eventhouse for event analytics, Warehouse for SQL analytics, and Lakehouse for engineering and mixed-data workloads.
Prefer Warehouse when a SQL team owns dimensional models, financial reporting structures, or repeatable T-SQL publication. Prefer Lakehouse when Spark, feature engineering, mixed formats, or flexible data preparation dominate. Prefer Eventhouse when event-scale investigation and time-series queries are the central workload.
A P&C carrier can prepare claim documents in a lakehouse, publish financial facts through a warehouse, and investigate operational signals in Eventhouse. That does not require replacing its claims system with a Fabric operational database. Keep the system of record where its transactional requirements are properly met.
Zero-unmanaged-copy ingestion: avoid unnecessary machinery
Zero copy is not a ban on copies. A maintained materialization can improve security, performance, reproducibility, or recovery. The problem is a copy nobody owns or can explain.
Use OneLake shortcuts to reference an already managed dataset. Use mirroring for supported, continuously replicated operational data. Use a Copy job or pipeline when the movement needs a supported batch, incremental, or orchestrated process. A mirror supplies a source-aligned representation; it does not automatically create the business history, reconciliation, or recovery contract downstream consumers require.
Use the least elaborate managed pattern that meets the source contract. A custodian feed already maintained as an approved table should not acquire three staging copies simply to cross team boundaries. A reinsurance feed requiring retained, reproducible evidence may justify a governed snapshot.
An illustrative design conversation captures the distinction: “Can we report directly from the mirror?” Yes, for some source-aligned questions. No, not as a substitute for agreeing what exposure, payment, or settlement means.
File ingestion: shortcut transformations or file-backed MLVs
The Pipeline You Don’t Have to Build argues against treating ingestion plumbing as an achievement in itself. The current choices make that argument stronger.
Shortcut transformations provide a managed file-to-Delta route. Select the transformation for the actual format and supported behavior; the release register still marks the Excel extension as preview, so do not apply the core capability’s status indiscriminately to every format.
Microsoft now also documents file-backed MLVs over OneLake folders, folder shortcuts, and individual CSV or Parquet files. The MLV SQL reference is the implementation starting point. This route can incorporate shaping and validation into the managed materialization instead of requiring an intermediate load notebook.
Choose shortcut transformations for straightforward managed conversion. Choose a file-backed MLV when declarative ingestion and transformation should be one maintained dependency. For a bordereaux feed, retain file provenance, identify malformed records, and reconcile totals before publication. Do not silently drop records to make a dashboard look clean.
File discovery, nested shortcut folders, schema drift, and refresh behavior are part of acceptance testing. A file being reachable in OneLake is not proof that a recurring refresh will discover every later arrival.
Materialized Lake Views: declare reusable transformation
Materialized Lake Views persist transformation results as Delta tables and provide dependency-aware refresh and monitoring. That makes them a strong default for recurring joins, conformance, and aggregates whose logic fits the supported declarative model.
Try an MLV before writing another custom transformation notebook. Do not turn that preference into a prohibition on code. API calls, iterative algorithms, model inference, and procedural processing still belong in appropriate notebook or application execution paths. Dataflow Gen2 remains useful when Power Query ownership and low-code preparation fit the team.
The optimal-refresh guidance is essential reading. Incremental processing requires suitable source configuration, including Delta change data feed, and supported query behavior. “Managed refresh” does not mean every workload always refreshes incrementally.
Apply data quality as code. A loan-performance product should expose missing account keys and invalid dates as failures requiring an owner. For financial reconciliation, I would ordinarily fail publication or route exceptions for review rather than discard unexplained balances.
History: preserve the question, not just the latest row
A changing source and a historical product are different assets. A current policy record cannot, by itself, explain what the insurer believed when it approved a claim.
Copy job CDC supports change processing and, for supported scenarios, SCD Type 2 history. Evaluate that managed route before building a general-purpose history framework. For questions involving both business-effective time and when the platform learned a fact, the bitemporal pattern remains a separate modeling responsibility.
Use explicit history when restatements, corrections, retrospective decisions, or audit evidence matter. For a backdated policy endorsement, preserve both the effective change and the date it became known. Document retention and reproducibility independently of a technical time-travel window. Backups, historical business records, and a test Known Copy solve different problems.
Semantic serving: Direct Lake by evaluation, not ideology
Direct Lake deserves first evaluation for strategic BI over curated Delta products. It uses the semantic model for relationships and measures while avoiding a conventional full data-import refresh. It still requires memory, capacity planning, and a well-designed model.
Direct Lake on OneLake can span multiple compatible Fabric data sources and does not fall back through the SQL analytics endpoint to DirectQuery. Direct Lake on SQL has different source and fallback behavior. That distinction belongs in the design, especially when SQL views or granular SQL security are involved.
Use Direct Lake when the governed data already exists in a compatible form and testing confirms the model’s behavior. Keep Import when its size, supported features, self-service workflow, or predictable refresh model makes it simpler. Keep DirectQuery when its source-side execution contract is required. “Never use Import” is not a useful architecture standard.
For wealth management, measure household assets, flows, fees, and market movement in an approved semantic model rather than asking every report or agent to recalculate them. Benchmark realistic concurrency and representative security roles. A fast administrator’s query does not establish an advisor’s production experience.
Security: choose the identity for every access path
OneLake Security belongs in the initial design wherever several supported engines consume the same data. It does not erase workspace privileges, SQL identity modes, or semantic-model permissions. An architecture must explain what the user or application can actually reach.
For shared lake access, evaluate the producer and consumer permissions in the shortcut security model. Passthrough and delegated authentication are different distribution designs; the delegated OneLake capability has its own preview status. Likewise, SQL endpoint identity modes are not interchangeable security settings.
There is also a legitimate fixed-identity Direct Lake pattern: report consumers use a secured semantic model while remaining unable to open the underlying lake data. Microsoft’s Direct Lake security guidance explains the relevant identity choices.
Choose shared data-layer security for multi-engine consumers, or a deliberately sealed semantic interface for report-only consumers. Test both allowed and denied access. A model’s row filters do not protect an independently granted raw-table path.
Fabric Policies: govern permitted actions, not business meaning
Fabric Policies are in preview. They provide scoped rules for selected platform actions, including external sharing, item creation, and editing workspace settings. They complement access control; they do not replace row-level security or decide whether an insurance payment is authorized.
Use this pattern to delegate within an approved operating envelope. A lending platform could allow an engineering group to create selected item types in a production capacity while limiting the same actions for other users.
My recommendation is controlled evaluation before broad enforcement. Conditional allow rules can have wider consequences than their author expects. Test included users, excluded users, unrelated workloads, and region support. An organization that delegates ownership needs an equally explicit answer to what has not been delegated.
Business Events: publish a condition that another team can use
A row changing is a technical event. A settlement variance requiring investigation is a business event. The latter carries a definition, an owner, and enough evidence for another team to respond.
Business Events are generally available, with individual publishing and consumption integrations retaining their own release boundaries. Define the contract through the Schema Registry, then let independent consumers implement their responses.
Use this pattern when a recurring analytical finding should initiate work. An illustrative Cards.SettlementVarianceDetected event might identify the processor, settlement batch, currency, variance, policy version, and evidence location. Operations opens a case; Finance investigates reconciliation; an analytical consumer measures resolution time. The producer should not need a new branch for each subscriber.
The delivery guarantees require consumers to tolerate repeated and reordered events. Use stable identifiers, deduplication, repeat-safe business operations, and reconciliation after interruptions. A schema version does not automatically make a changed payload compatible with every consumer.
Choose the detection interval from the business response window. Daily reinsurance reconciliation does not become more valuable merely because its last hop streams. Seconds may matter for card operations; they are not a universal requirement. The event describes a condition, not permission to move money.
Capacity operations: protect the service the business depends on
The Fabric capacity control-loop pattern connects telemetry to an operating response. Observe what happened, identify the affected service, act within an approved policy, and measure recovery.
Capacity-level surge protection can reject new background work before deeper throttling. It does not terminate existing jobs or guarantee uninterrupted interactive service. Workspace-level surge protection is a separate preview capability.
October’s semantic-model consumption reporting improvement makes in-progress background usage visible sooner; Microsoft describes it as automatic. Set thresholds from observed workload behavior, and give rejected work a retry and recovery path. A failed refresh should not disappear into an operations report nobody reads.
Capacity overage is another decision: paying for eligible excess consumption differs from declining work. Assign a cost owner and explicit limits rather than letting the commercial choice remain implicit.
Use dedicated capacity when the consequence of contention justifies isolation. A carrier’s regulatory reporting window should not depend on the good behavior of an unrelated experimental workload. Use capacity telemetry alongside the estate-wide ownership and usage perspective described in the EduDataSci FUAM baseline.
CI/CD and objects as code: prove the target is usable
Definitions Move. Data Stays is the organizing principle. Moving a lakehouse item is not the same as recreating its internal objects, populating its data, or proving that its consumers have the right permissions.
Use Fabric CI/CD to combine version control, controlled promotion, environment-specific configuration, and automated verification. Check item-level support rather than assuming every feature follows an identical deployment path. Deployment plans provide additional ordering and pre/post actions, but their preview status should remain visible in the design.
Use an idempotent configuration step to establish required structure, then apply deliberate migrations. Create missing schemas and required objects safely. Do not blindly replace populated tables to make a deployment repeatable. Validate shortcut targets, active configuration, semantic-model bindings, ownership, access, and representative results after promotion.
For a lending product, the useful release evidence is not simply “deployment succeeded.” It is “this version passed against the approved test baseline, resolves to the intended production surfaces, and returns the expected result under the intended consumer identity.”
That is the difference between moving artifacts and releasing a working product.
Agent-ready meaning and authorized applications
One Agent, Several Kinds of Truth distinguishes a current balance, the definition of balance, and permission to change it. Keep those authorities separate as Fabric gains more conversational and application experiences.
Fabric data agents provide governed conversational querying. Current documentation supports up to five sources in a combination that can include multiple semantic models. Their data queries remain read-only. A data agent is not a write-back endpoint or a replacement for transaction authorization.
Start with a bounded decision domain and known questions. A wealth agent should use approved measures to distinguish market movement from net flows, not invent its own definition of growth. Evaluate answers, source selection, access boundaries, and regressions as models and instructions evolve.
Fabric IQ ontology remains preview. Use it where entities, relationships, and source-owned measures need common meaning across products. Do not require a full ontology before publishing a useful semantic model. Existing pilots should also review Microsoft’s migration notice: the old ontology experience is scheduled to retire on January 31, 2027.
Fabric Apps and App functions are preview options for bringing a supported application closer to governed data. Functions add server-side logic and downstream calls using the application’s identity. That makes application-level validation essential: authenticating the application does not prove that its current user may perform every operation the application can perform.
A claims assistant might explain why a case needs specialist review. A separately authorized service can create the referral and record the outcome. I would keep that boundary explicit even when the user sees one seamless experience.
Put the patterns together around one business outcome
Consider a card-settlement exception product. Its landing boundary retains protected processor data and publishes minimized development inputs. A named test baseline preserves known reconciliation cases. Managed ingestion feeds declarative transformation where supported; the published surface defines the settlement grain and currency treatment.
A semantic model supplies approved variance measures. A business event announces a qualifying exception. A read-only agent explains the evidence. An authorized case-management service records the response. Capacity controls protect the operating window, and the release pipeline validates the whole path under representative identities.
The outcome is not “we used every Fabric workload.” It is a repeatable way to identify, explain, route, and resolve a business problem while preserving the evidence behind it.
That is the baseline I recommend: protected arrival, explicit transformation, stable publication, deliberate identity, observable operation, and controlled change. Add previews where their value justifies evaluation, without making them an invisible dependency of an essential service.
Take an important Fabric workload and ask what it promises, who owns that promise, and which pattern enforces it. If those answers are clear, the next feature becomes an informed implementation choice. If they are not, another feature will only make the uncertainty harder to see.