Strategy – Page 2 – EduDataSci – Educating the world about data and leadership

Stop Picking a “Winner”: Data Product Interoperability Between Databricks and Fabric in Financial Services

Financial services teams have a familiar argument: “Are we a Databricks shop or a Fabric shop?” It sounds like a strategic question, but it usually hides the real problem—different parts of the business need different ways to use the same data, under tight controls, with clear auditability.

When Databricks and Microsoft Fabric interoperate at the data product level, the conversation shifts from which platform to how the data must be used: BI and semantic models, heavy Spark engineering, real-time analytics, governed sharing across boundaries, or advanced ML. The platform becomes a means, not the decision.

In this post I’ll lay out what “data product level interoperability” looks like in practice, why it enables responsible best-of-breed choices in regulated environments, and how it plays out in both directions: Databricks → Fabric and Fabric → Databricks.

When Facts Don’t Live in a Domain: Why Data Products Beat Pure Domain-Driven Data Engineering

There’s a pattern I see in mature analytics organizations: as soon as the data platform gets big enough to feel “enterprise,” someone reaches for domain-driven design (DDD) as the organizing principle for data engineering and governance.

It’s an understandable move. DDD gives us language for ownership, boundaries, and “this team is responsible for that thing.” And when you’re trying to untangle a spaghetti warehouse, that sounds like oxygen.

But here’s the catch: the most valuable data in a warehouse—the fact tables in traditional star schemas—often doesn’t live neatly inside a single domain. It lives at the intersections.

In this post, I’m going to make three points:

why domain boundaries tend to break down precisely where the business value shows up (facts)
how a data product model handles “intersection data” without inventing reference domains and mega-joins
why Microsoft Fabric’s workspace + lakehouse + OneLake tooling makes the product approach easier to execute in practice (especially with shortcuts, security, and materialized lake views)

And yes—domains still matter. Just not as the unit of delivery.

2025 Year in Review: When Microsoft Fabric and Microsoft Purview Turned “Data + AI” Into a Governed Operating Model

By the end of 2025, the conversation around analytics stopped being about dashboards and started sounding a lot more like operations. The rise of autonomous and semi-autonomous agents put a sharper edge on an old truth: AI only becomes an enterprise capability when the underlying data is trusted, discoverable, and defensible.

Microsoft Fabric and Microsoft Purview spent 2025 building toward that reality from opposite (but increasingly overlapping) sides of the house. Fabric pushed the platform forward—unifying workloads, expanding OneLake, and adding new intelligence and database capabilities designed for AI-era workloads. Purview tightened the governance and security loop—making data quality, cataloging, risk visibility, and policy enforcement feel less like a separate initiative and more like part of the daily flow.

This year-in-review walks through the story they told together: how Microsoft Fabric expanded the data platform into a more complete data estate, how Microsoft Purview reframed governance for the AI and agent era, and where the two converged into a more practical, enterprise-ready operating model.

From Warehouses to Products: SAMR for Your Cloud Data Platform

Financial Services, Insurance, Wealth Management, and Professional Services have a gift—and a curse—when it comes to data.

The gift is that these industries know how to run critical systems with discipline. The curse is that we’re so good at controlling risk that we often rebuild the same constraints in every new platform we adopt.

That’s why so many “modern cloud data platforms” in these sectors end up feeling like the old data warehouse with a new hosting model: better infrastructure, familiar bottlenecks. Slow change. Long planning cycles. A platform measured by what it ingests, not what it enables.

In the previous post on AI and SAMR, the point wasn’t that AI is inherently transformative. The point was that without a disciplined framework, we use new tools to reproduce old workflows. The same is true for data.

Here’s what I’ll cover in this follow-up:

Why cloud platforms in regulated, risk-sensitive industries so often replicate warehouse-era behaviors.
How SAMR can be used as a simple truth-teller for your data platform strategy.
Why data products are the most practical “unit of redefinition,” and why you don’t need Data Mesh to benefit from them—though mesh and product thinking dramatically expands your toolset.
How ground-up design thinking keeps your platform anchored to real business goals, not elegant architectures

If your cloud platform still takes six months to change a definition, you didn’t modernize. You relocated.

From “Should‑Do” to Done: Digital Workers for Wealth, Energy, and Financial Services

Every enterprise carries a shadow backlog—the should‑do work that never beats the urgent. It’s the reconciliation that almost closes, the control that’s “fine for now,” the evidence that exists but isn’t filed where audit will accept it. None of these items is existential in isolation; together they become trust debt: silent risk, rework, slower decisions, and reputational drag.

2025 amplified the problem. Compressed settlement cycles demand same‑day precision in wealth operations. Emissions and operational reporting remain high‑stakes in oil and gas, with timelines adjusted but scrutiny intact. Financial institutions face fluid sanctions and evolving data‑sharing rules, while beneficial ownership reporting shifted materially. In each case, the gap is less about intent and more about capacity. Digital workers exist to close that gap.

Digital workers are policy‑aware software agents that connect to the systems already in place, act on explicit rules (with narrow ML where safe), and hand back proof that the job was completed—consistently and auditable. What follows clarifies how they work and where they quietly create value in wealth management, oil and gas, and financial services.

Spec‑Driven Development: Make the Specification the First Commit

If your acceptance criteria live in a comment thread, they’re not requirements—they’re opinions. Spec‑driven development (SDD) turns those opinions into executable truth so code, tests, docs, and operations move in lockstep.

Building on our split between functional and nonfunctional requirements, this follow‑up introduces spec‑driven development: what it is, why it reduces drift, and how to run it inside agile without ceremony. We’ll connect behavior specs, API contracts, data schemas, and quality budgets to lightweight gates in CI/CD and SLOs in production. By the end, you’ll have a “small slice” pattern you can ship next sprint. This is where spec-driven development meets Agile, DevOps, and APIs.

From Substitution to Outcomes: How AI and SAMR Are Forcing a Rethink of Development Strategy

We like to say we’ve “transformed” how work gets done. But if you look closely at many enterprise systems, you still see the outline of a paper form hiding under a slick UI.

We replaced paper with terminals, terminals with web apps, web apps with SaaS—and then pointed automation at the whole stack. In too many places, we’ve simply substituted one medium for another, without asking whether the underlying process still makes sense.

In this post, I’ll do three things:

Introduce the SAMR framework as a way to think about how technology shapes work.
Show how many business processes sit on layers of substitution going all the way back to paper.
Explain how AI enables goal‑based processes, where we define the outcome and how we’ll know we’ve met it, and let AI figure out the path—without prescriptive step‑by‑step code.

And along the way, we’ll confront an uncomfortable idea: some of your “modern” processes may still be driven by the personal preferences of someone who retired more than fifty years ago.

Agile Needs a Spine: Aligning Dates, Deliverables, Objectives, and KPIs

A recent conversation with a colleague reminded me of how important applying structure to an agile project team is. Most industries can’t take, to quote the old Blizzard line, the option to release it “when it’s ready,” because they’ve made commitments to customers or other parts of the business.

Agile frees teams from big-batch planning, but it doesn’t free them from consequences. When dates, deliverables, objectives, and KPIs drift apart, you get motion without momentum—busy sprints, delayed value, and vague success. In this piece, I’ll show why alignment across these four anchors is the operating system for agility, how to keep it lightweight, and why you can’t—and shouldn’t—abandon traditional project management entirely.

Data Literacy, Citizen Analysis, and the Shift to a Data‑Enabled Culture

I’ve never been fond of the phrase data‑driven. It can imply that people should surrender the wheel to whatever the chart says. I prefer data‑enabled: a culture where evidence is visible, disputable, and useful—where humans steer and data is the headlight, not the driver. That shift doesn’t start with a platform; it starts with literacy, and it grows when more people can do a little analysis for themselves.

Beyond Boxes and Lines: Designing for the System of Systems Your Software Must Live In

Architecture diagrams look clean—until they collide with the place your system actually lives. That place is not just “production.” It’s your organization: a dense system of systems made of teams, processes, policies, budgets, and tools. The primary reason to understand the organization in which a system will reside is precisely these systems. Your software must integrate with them as surely as it integrates with databases and queues.

I’ll make one argument three ways. First, I’ll define system of systems and show why the org is part of the runtime. Next, I’ll connect business architecture to systems architecture so strategy and structure reinforce each other. Finally, I’ll revisit Conway’s Law—how to use it deliberately and how it can quietly work against you. Then I’ll close with a practical loop you can run before you write code.