Architecture – EduDataSci – Educating the world about data and leadership

The GPU Is Not the Product: What Query Acceleration Changes in Microsoft Fabric

The moment a warehouse most needs predictable performance is usually the moment everyone asks it a question at once.

Market volatility drives simultaneous risk, liquidity and exposure queries. A card-processing incident sends analysts, applications and investigators into the same transaction history. Month-end reporting puts Finance, Operations and executives on the same governed data while automated services continue doing their own work in the background.

Microsoft Fabric’s new query acceleration preview addresses that pressure in an unusual way. The T-SQL does not change. The tables do not move. Fabric identifies eligible scans, joins and aggregations and sends them to a GPU working beside the CPU inside the warehouse execution plan.

That does not make every query seven times faster or add general-purpose GPU compute to Fabric notebooks. It makes specialized compute an internal property of the governed data product, below the interface its consumers already use.

The Landing Lakehouse: The Fabric Pattern Between Arrival and Use

A source begins mirroring into Microsoft Fabric on Friday. By Monday, the tables are visible in OneLake. Production engineers can build from them. Developers can query them. Testers can point validation jobs at them. Analysts can begin exploring.

The delivery looks successful because the data arrived quickly. The architecture may already be in trouble.

The first reachable version of data is rarely the version every user and environment should consume. Production needs fidelity and controlled change. Development needs realistic relationships without unnecessary exposure. Test needs stable inputs that do not shift underneath automated validation. The source team needs to retain authority over what lands and what may be published. Governance needs a defensible answer to a deceptively simple question: which version of this data is visible to whom?

A Landing Lakehouse provides that answer.

The pattern uses a dedicated Lakehouse as the controlled boundary between direct arrival and governed use. Mirrored data, external shortcuts, and source-owned Fabric data can land once. The Landing Lakehouse then exposes distinct schemas for production, anonymized development, representative subsets, Known Copies, and curated validation. Each output is a separate surface with its own purpose, access model, refresh behavior, and owner.

The point is not to add another layer for its own sake. The point is to keep the convenience of Fabric from collapsing several different contracts into one raw surface.

When Software Becomes Disposable: Why the Next Data Mesh Needs Action-Oriented Data Products

Something important changes when intent becomes the front door to software. If a banker, underwriter, advisor, claims lead, or merchant-risk analyst can ask for a task-specific application and receive it in minutes, then bespoke software stops being rare. It becomes cheap. The expensive thing is no longer the application layer. The expensive thing is the interface to truth: the boundary where customer data, authorization, business policy, and systems of record meet.

That shift deserves a different architectural response. I like to frame a data product as reusable, trustworthy, contract-driven, loosely coupled asset with clear ownership, explicit promises about freshness and meaning, and a versioned surface consumers can rely on. The data mesh tradition adds domain ownership, self-serve platform capability, and federated computational governance. My ingest-transform-surface model sharpens the picture further: inputs are explicit, transformation is deliberate, and the published surface is the contract. All of that is right. It just is not enough once agents and intent-driven tools begin to mediate real work.

Domain APIs need to become the front door for fit-for-purpose apps, and the mesh becomes the process glue that turns operational data into analyzable insight and analytical insight into operational action. My argument is that we should now make that shift explicit. The next mesh should be built from Action-Oriented Data Products: domain-owned data products whose interfaces declare not only what can be read and analyzed, but what can be done, by whom, for what purpose, with which guardrails, and against which authoritative source.

This is not a new name for old API management. It is a claim about the next architectural quantum. Data mesh already argued that the data product is the quantum of the analytical estate. In an intent-mediated world, that quantum must become action-oriented. The mesh stays. The node changes. To make that case, I want to do three things: show why current data products stop short, explain the historical parallels to object orientation, microservices, and data mesh, and define the architecture that lets agents discover and manipulate data safely.

One Agent, Several Kinds of Truth

An enterprise agent rarely fails because it cannot produce a fluent paragraph. It fails because it cannot distinguish the current balance from the definition of balance, the approved policy from the email that discussed an exception, or the authority to explain a decision from the authority to change a system of record.

That distinction matters in financial services. A relationship manager asking why a commercial borrower has become riskier is really asking several questions at once. What changed in the borrower’s accounts? How does the exposure roll through facilities, collateral, guarantees, and legal entities? What does credit policy require? What did the deal team agree to last week? If action is warranted, who may open a review, request documents, or change a limit?

No single knowledge source can answer all of that honestly. The useful architecture is a Copilot Studio agent that coordinates several bounded kinds of intelligence: Fabric IQ for business state, Foundry IQ for institutional knowledge, Work IQ for the user’s working context, an ontology for shared meaning, and MCP tools for controlled action. API Center anchors design-time inventory and discovery, while Microsoft Purview anchors data governance and supported AI-interaction compliance. Runtime enforcement remains with Entra, Power Platform data policies, native IQ permissions, API Management, and the source systems.

Choose the Publication Contract, Not the Fabric Icon

A bank’s credit-risk reporting team needs one change before the monthly committee pack. The underlying exposure data is current. The semantic model can reach it. The requested change—reclassifying a set of delinquency bands—is not particularly difficult.

Yet the team that owns the report cannot publish the change. The curated table is maintained by a Spark process owned elsewhere, so the request becomes a ticket, a handoff, and a negotiation over another team’s release calendar. Nothing in Microsoft Fabric is broken. The architecture treated the point where data was published as a storage destination when it was really a contract with an owner.

The reverse failure is just as common. A data-engineering team already publishes well-governed Delta tables for analytics, data science, and downstream domains, but an architectural standard requires every Gold table to be copied into a Warehouse before Power BI can use it. The second published copy adds another load, another security boundary, another incident path, and another opportunity for the two versions to disagree. It exists because a diagram said Gold should look like a database.

Fabric Warehouse and Lakehouse overlap too much for the old rule—structured data goes to a warehouse; everything else goes to a lakehouse—to make this decision well. Both use Delta in OneLake. Both can serve SQL queries. Both can support Power BI and governed analytical products. The consequential difference is the publication contract each item is being asked to own and the team expected to keep it.

Choose a Lakehouse publication contract when the authoritative product is maintained as Delta through engineering workflows and SQL consumers primarily need to read it. Choose a Warehouse publication contract when the owning team must create, populate, correct, secure, and release a relational product through T-SQL, especially when multi-table transactions or an independent SQL-owned release cadence matter. Use both when two real contracts exist—not because one Fabric icon feels unfinished without the other.

The Workspace Is Not the Product: Fabric Objects as Code and the Rise of Lakehouse as Code

A Microsoft Fabric deployment can finish successfully and still leave you with an empty Lakehouse.

The notebook arrives. The pipeline arrives. The Lakehouse item appears in the target workspace. References are rebound to the correct environment. Every deployment task is green. Then the team opens the Lakehouse and discovers that the schemas, Delta tables, Spark views, and materialized lake views expected by the solution are not there.

That outcome is not a contradiction. It exposes an architectural boundary that Fabric teams need to make explicit: deploying a Fabric item is not the same as constructing the state inside that item.

The distinction matters in any enterprise. It matters more in financial services, where a deployment may support card authorization controls, wealth positions, settlement operations, lending decisions, or insurance exposure reporting. A release that creates the application shell without the required internal structures has not delivered a usable data product, however successful the pipeline log may look.

This article updates the idea of Fabric objects as code for the platform Microsoft is shipping in 2026. Fabric now has a much more coherent delivery stack: Terraform for stable platform resources, Git-backed item definitions, Variable Libraries for configuration, fabric-cicd and Fabric CLI for scripted deployment, and materialized lake views for declarative Lakehouse transformations. The result is not one universal deployment tool. It is a layered architecture in which each form of state is managed by the mechanism suited to it.

When Everyone Can Build, the Specification Becomes the Control Plane

One of the easiest mistakes in the current AI cycle is to treat vibe coding as a developer-productivity story.

It is certainly that. A developer can use an agent to explore a repository, create a plan, write code, run tests, and open a pull request. But the more important change is happening outside the traditional development organization. A claims analyst, credit-policy manager, operations lead, product owner, or adviser-support team can increasingly describe a problem and receive a working application, automation, or analytical tool in return.

OpenAI reported in June 2026 that knowledge workers represented about 20 percent of Codex users and were increasingly using it to build lightweight tools that previously required engineering support. That is one vendor’s usage data, not a census of enterprise development, but it captures the direction: the boundary between understanding a business problem and being able to build software around it is getting thinner. OpenAI: Codex for knowledge work

That changes the control problem.

The old governance model assumed that code entered through an engineering workflow, where developers, architects, security teams, and reviewers could resolve what the requirements left unclear. Democratized development weakens that assumption. The answer is not to force every useful idea back through the old bottleneck. It is to move the primary control point earlier—from reviewing code after it exists to governing the specification that tells people and agents what may be built.

When software creation is democratized, the specification becomes the enterprise control plane.

This post explains why spec-driven development has become much more important, what changed in the tooling during 2025 and 2026, and what a fully integrated enterprise specification should contain. The goal is not more paperwork. It is executable clarity before autonomous execution.

Stop Publishing the Workshop: An Ingest–Transform–Surface Architecture for Microsoft Fabric

Most Microsoft Fabric architecture diagrams are too polite.

They show data arriving, becoming cleaner, and eventually appearing in a report. What they usually leave out is the argument that determines whether the platform remains understandable after the third development team, the tenth source system, and the fiftieth downstream consumer: where does source reality end, where does product construction happen, and where does the enterprise make a promise?

When those boundaries are unclear, every table is treated as equally public. Analysts connect to whichever object is easiest to find, engineers preserve internal tables because somebody may be using them, and source-facing objects begin to look like product interfaces. The platform may still run, but the architecture has stopped communicating intent.

This article proposes a deliberately simple alternative for a schema-enabled Fabric Lakehouse:

Every ingestion method lands data in, or exposes it through, ingest.*.
Materialized Lake Views (MLVs) and notebooks move data into transform.*.
MLVs publish stable, consumption-ready tables into surface.*.
Consumers connect to the product through governed interfaces built on surface.*, not through the internal construction layers.

Here, ingest.* is shorthand for every table in the ingest schema. The same applies to transform.* and surface.*; the asterisk is architectural notation, not part of the schema name.

The result is not another three-color diagram. It is a practical way to separate source capture, internal implementation, and the published contract of a data product.

Stop Thinking in Rows: Databases Make More Sense as Sets

A database table looks like a spreadsheet. That resemblance is useful—and damaging. It makes tables approachable, but encourages a procedural picture: start at the top, inspect a row, make a decision, and move on. That is usually the wrong mental model.

Rows still matter. They have keys, carry facts, and occasionally ruin your afternoon. But the more useful unit of thought is the set: the population of tuples that should exist at a particular stage of processing.

With that shift, SQL operators construct populations. This post develops that mental model, then recasts the EduDataSci freeze-and-squash pattern for a Fabric MLV change feed as small set operations.

The tuple is the unit of assertion; the set is the unit of thought

Suppose a lender receives a monthly snapshot of every commercial loan. “One row per loan per month” states the grain of loan_snapshot, but not its meaning. A better description is:

loan_snapshot is the set of loan-state observations supplied by the servicing system, keyed by loan and snapshot date.

Each tuple asserts a loan’s balance, servicing status, risk grade, and other attributes at a time. “Find the delinquent loans” now defines a subset rather than a row-by-row flagging process. “Find loans missing from the new complete snapshot” defines the difference between two key sets. The database decides how to produce those populations; our first job is to define them correctly.

The Internet After Apps: Public, Private, and Personal AODPs in an Intent-Driven World

When software can be generated for a person, a task, or a moment, the durable architecture must move out of the application and into the interface to data and action.

A wealth advisor finishes a client call and asks an agent to build a small application.

The application should assemble the household’s accounts, identify securities that could raise $75,000, preserve the client’s income targets, avoid restricted positions, estimate tax consequences, and prepare an explanation for the client. It is not intended to become the next enterprise advisor platform. It may only be needed for the next hour.

The software could be generated almost immediately. That is the appealing part.

The dangerous part is everything the software must touch.

It needs market information, client preferences, account positions, tax lots, suitability rules, firm policies, trading restrictions, approval thresholds, and access to an order-management system. It must know which information belongs to the client, which belongs to the firm, and which comes from public or commercial sources. It must distinguish between calculating a proposed trade and submitting one. It must preserve evidence of what it read, why it made a recommendation, who approved it, and what eventually changed.

The application may be disposable. Its authority cannot be.

This is the architectural problem hiding underneath intent-driven computing and enterprise vibe coding. As software becomes easier to generate, more of the burden moves onto the interfaces through which that software reaches data, policy, and operational systems. We cannot make every dynamically generated application trustworthy in the same way that we certify a major enterprise platform. We can, however, force every application to operate through trustworthy products.

That is the purpose of the Action-Oriented Data Product, or AODP.

An AODP is a reusable, domain-owned product that exposes governed data and governed actions through a secure, discoverable, contract-driven interface. AODPs can exist in public, private, and personal spaces. Connected through a mesh, they provide the durable substrate underneath people, agents, and software assembled for individual tasks.

The internet built around pages and applications does not disappear. It gains another layer: an internet of governed data and action.