Non-Tech Thursday – EduDataSci – Educating the world about data and leadership

Testing Like We Mean It: Bringing Software‑Grade Discipline to Data Engineering

I like to say that the first product of a data team isn’t a table or a dashboard—it’s trust. Trust is built the same way in data as it is in software: through tests that catch regressions, encode intent, and make change safe. If pipelines are code, then they deserve the same rigor as code. That means unit tests you can run in seconds, integration tests that respect the messy edges of reality, comprehensive tests that exercise the platform end‑to‑end, and user acceptance testing that proves the system answers the questions people actually have. Done well, this isn’t busywork; it’s the backbone of reliability and a pillar of governance.

Conway’s Law for Data Teams

Two Dashboards, One Truth

On Monday, Maya—head of a seven‑person data team—watched two dashboards disagree.

The executive dashboard showed $11.2M in MRR. Sales’ dashboard said $10.6M. Both pulled from “the warehouse.” Both refreshed nightly. Neither was “wrong”; they just measured different things.

Maya didn’t control how Sales Ops or Marketing were organized, who they reported to, or which tools they bought. She controlled only her data team—its models, interfaces, and operations. Yet the warehouse had clearly taken on the shape of the company’s communication patterns.

Conway’s Law, without asking permission, had moved in.

Improvement Science for Business Leaders: A Practical Playbook for Better, Faster Results

Most executives know Lean, Six Sigma, and Agile. Improvement science is the disciplined backbone behind those methods—a way to get measurable gains by learning quickly in the real world, not just in the boardroom. It’s been refined for decades in healthcare and education, but its core ideas translate cleanly to sales, operations, CX, finance, HR, and product. Here’s what it is—and how to start using it immediately.