Microsoft Fabric – Page 4 – EduDataSci – Educating the world about data and leadership

DirectLake on OneLake CI/CD: A Practical Two-Step Deployment Pattern with Sempy Labs + Variable Libraries

DirectLake on OneLake is one of those “this is what we’ve been waiting for” features in Microsoft Fabric—until you try to deploy it cleanly across Dev → Test → Prod and realize you’ve re-entered the world of post-deployment manual fixes.

In this how-to, I’m going to do three things:

Contrast DirectLake on SQL endpoints (the “classic” flavor) with DirectLake on OneLake (the newer flavor), and explain why OneLake is worth the trouble.
Walk through the normal deployment pipeline approach that works well for DirectLake on SQL.
Show a two-step, semi-automated approach for DirectLake on OneLake using:
- sempy_labs.directlake.update_direct_lake_model_connection(), and
- a Fabric Variable Library + a “run-after-deployment” notebook.

Along the way, I’ll call out the current challenges (because yes, they’re real right now), and why this pattern matters for serious Microsoft Fabric Power BI CI/CD work.

Two Flavors of DirectLake: Over SQL vs. Over OneLake (and How to Switch Without Surprises)

DirectLake has a way of sounding wonderfully simple: “Power BI, but it reads the lake directly.” Then you build two semantic models that both say DirectLake, and they behave… differently. One falls back to DirectQuery when you least expect it. Another refuses to touch your SQL views. Security works for you, but not for your report consumers. Suddenly, “DirectLake” feels less like a feature and more like a riddle.

The good news: this is explainable. And once you understand the two flavors—DirectLake over SQL and DirectLake over OneLake—you can choose deliberately, design around the trade-offs, and even switch between them when you have to.

In this post, I’ll demystify what each option really means, lay out the positives and negatives, explain when you’d use each (and why), and show how to switch using Semantic Link Labs—including what can break when you flip the switch.

From Tables to Networks: A Deep Dive into Graph in Microsoft Fabric for Financial Services Insights

Most financial services data is already “connected.” It just isn’t modeled that way.

Fraud rings don’t show up as a single row. Money laundering doesn’t announce itself in one transaction. Counterparty exposure isn’t obvious from one booking. The meaningful signal lives in relationships: who shares an address, which accounts route funds through the same nodes, where devices and identities overlap, and how risk propagates through a network.

Graph in Microsoft Fabric is designed for exactly that: turning your OneLake data into a connected model you can explore visually, query with GQL, and enrich with built-in graph algorithms—without standing up a separate graph stack and duplicating data.

In financial services, this is the difference between “we have the data” and “we can reason over the connections.”

Gold as the Contract: Schema Evolution, Data Products, and Governance in Microsoft Fabric

A schema change is rarely “just a schema change.”

It’s the moment an upstream team’s perfectly reasonable adjustment becomes a downstream team’s broken report, confusing metric, or silent misinterpretation. And that’s why schema evolution has always been a source of anxiety: a schema isn’t simply structure—it’s an interface.

In this post, I’ll do three things. First, I’ll ground why schema evolution has historically been such a persistent concern. Next, I’ll reframe Medallion with Gold as the published surface area of the data product, and Silver as an optional workshop layer where data is supplemented and transformed. Finally, I’ll connect that design to how Microsoft Fabric supports it today—especially the post‑Ignite direction around governance, security, and semantics.

Power BI Copilot Has Multiple Modes. Here’s What Each One Does—and How Fabric Data Agents Change the Game.

Copilot in Power BI isn’t “one feature.” It’s a growing set of experiences that show up in different places, behave differently, and—most importantly—solve different problems.

That’s why two people can both say “Copilot didn’t work for me,” and both be right. One might be trying to generate a report page in Desktop. Another might be trying to chat across any model in their tenant. A third might be expecting an agent-like experience that stays grounded in a curated subject area.

In this post, we’ll map the major modes of Power BI Copilot (where it shows up, what it’s best at, and what it’s not), then contrast that with Fabric Data Agents—because Data Agents aren’t a “better Copilot.” They’re a different building block, meant for a different kind of outcome.

From ClickOps to Confidence: CI/CD Best Practices for Microsoft Fabric

If you’ve ever clicked Deploy in Fabric, watched the spinner, and hoped nothing “mysteriously” changed in Test or Prod… you’re not alone.

Microsoft Fabric has made it possible to manage analytics artifacts like software. But getting to reliable releases—repeatable deployments, environment-safe configuration, and auditable changes—still takes intent. In this post, I’ll walk through a practical, production-minded approach to CI/CD in Microsoft Fabric: how to structure Deployment Pipelines, where Git fits, how Variable Libraries and Deployment Rules reduce environment drift, and when to lean on Fabric-CICD and APIs to move beyond the UI.

Materialize Responsibly: How Fabric’s External Data Materialization Affects “Zero Unmanaged Copy” — and Where Materialized Lake Views Now Shine

Microsoft Fabric’s Warehouse can now materialize external files into tables with straight‑ahead T‑SQL, and Materialized Lake Views (MLVs) have quietly leveled up with optimal refresh (including incremental) and stronger, UI‑backed monitoring. If your north star is Zero unmanaged copy, the question isn’t “should I materialize?”—it’s “how do I materialize responsibly under OneLake governance?” Here’s what changed since our last take—and what to use when.

Real‑Time Data Isn’t Free: The Complexity and Cost Tradeoffs (From Trickle to Internet‑Class)

The first time someone asks for “real‑time,” it sounds like a small tweak: refresh the dashboard faster, trigger an alert sooner, show a counter that feels alive. In a data platform, that single request quietly changes everything—how you ingest, how you process, how you serve, and how you operate.

This post keeps it practical. It frames real‑time as a freshness target (not a vibe), walks through the two taxes real‑time introduces—architectural complexity and cost—and shows how patterns evolve as you scale from modest #StreamingData to internet‑class velocity. It also folds in recent Microsoft Ignite announcements that matter for real‑time platforms, including SQL Server 2025’s “change event streaming” and near real‑time analytics via OneLake/Fabric mirroring, plus the continued maturation of Microsoft Fabric’s Real‑Time Intelligence building blocks.

2025 Year in Review: When Microsoft Fabric and Microsoft Purview Turned “Data + AI” Into a Governed Operating Model

By the end of 2025, the conversation around analytics stopped being about dashboards and started sounding a lot more like operations. The rise of autonomous and semi-autonomous agents put a sharper edge on an old truth: AI only becomes an enterprise capability when the underlying data is trusted, discoverable, and defensible.

Microsoft Fabric and Microsoft Purview spent 2025 building toward that reality from opposite (but increasingly overlapping) sides of the house. Fabric pushed the platform forward—unifying workloads, expanding OneLake, and adding new intelligence and database capabilities designed for AI-era workloads. Purview tightened the governance and security loop—making data quality, cataloging, risk visibility, and policy enforcement feel less like a separate initiative and more like part of the daily flow.

This year-in-review walks through the story they told together: how Microsoft Fabric expanded the data platform into a more complete data estate, how Microsoft Purview reframed governance for the AI and agent era, and where the two converged into a more practical, enterprise-ready operating model.

Edit, Retarget, and Redeploy: A Practical TMDL Folder Workflow for Fabric Semantic Models

There’s a moment in every Fabric semantic model lifecycle where the “click it in the UI” approach stops scaling.

It usually happens when you need to rename dozens (or hundreds) of fields to match a business glossary, or when Dev is stable and you’re ready to point the same model at a new Lakehouse for Test/Prod. That’s when the model stops being a diagram and starts being an artifact—something you want to treat like code.

This guide reflows the whole workflow end-to-end, using the Fabric service Edit in Desktop experience to open the model, exporting it to a PBIP project stored as a TMDL folder, editing that folder externally (no scripting inside Power BI Desktop), and then getting those changes back into the service—including the key capability you asked for:

retargeting the entire model to a different Lakehouse/Warehouse, and
retargeting a single table to a different physical table (even in a new Lakehouse).

We’ll do it with the mindset of Power BI + Microsoft Fabric development: repeatable changes, visible diffs, and fewer “hand edits” you regret later.