Pre‑Ignite 2025: The State of the Data Platform

What to watch in the eight days before Ignite—and how to translate the signals into architecture bets that age well.

The short version

This pre‑Ignite landscape scan argues that the data platform has converged around five themes: a unified lake‑centric control plane, real‑time by default, vector‑native databases, federated governance that actually works, and cost visibility as a first‑class feature. Microsoft’s stack—Fabric + OneLake + Azure data services—is moving fastest on all five. Expect Ignite (Nov 18–21, San Francisco + online) to sharpen these trajectories rather than reroute them.

1) The platform has converged on the lake

Fabric is now the organizing plane for analytics, with OneLake at the center and open table formats as the interoperability contract. Microsoft formally added Apache Iceberg support in OneLake, alongside Delta, and introduced OneLake Table APIs so external engines can treat OneLake as an Iceberg catalog—exactly the kind of “open compute over open tables” pattern enterprises have been asking for. Partnerships that enable bi‑directional access with Snowflake reinforce the trend: store once, query anywhere. 

OneLake’s shortcuts make the “store once” idea practical across clouds. Notably, S3‑compatible shortcuts can use file caching to reduce egress costs, with key/secret auth today (Entra OAuth not yet supported). On the consumption side, Power BI’s Direct Lake mode continues to mature as the default semantic access pattern for Delta tables in OneLake. Together, these moves stitch multicloud storage into a single analytic surface without brittle copy pipelines. 

2) Real‑time is no longer a special case

Fabric’s Real‑Time Intelligence (Eventstreams, KQL databases, Eventhouse) puts the Azure Data Explorer/Kusto engine inside the lakehouse era. The retirement of Synapse Data Explorer (preview) on Oct 7, 2025 and Microsoft’s guided migration to Eventhouse make the direction explicit: one real‑time stack, native to Fabric. Activator (the rules engine for triggering actions) rounds out an event‑to‑action loop inside the same platform. 

3) Databases have gone vector‑native

The most important database story of the year is quiet but foundational: vector becomes a first‑class citizen.

Azure SQL Database shipped a native VECTOR type and functions (GA, June 2025), and SQL Server 2025 (preview) adds vector indexes for approximate nearest‑neighbor search. This pushes RAG patterns into familiar SQL tooling rather than offloading to a separate store.  Azure Cosmos DB offers vector search (NoSQL) and hybrid search (vector + BM25), positioning operational data stores to participate in semantic retrieval without an extra hop. 

Taken together, you can now put embeddings beside the rows you already govern, back them with your existing backup/HA posture, and query them with your existing identity model. That simplification is the point.

4) Security and governance have shifted left—into the lake

Two capabilities are worth calling out because they move security controls closer to where data actually lives:

Workspace‑level Private Link is generally available, making granular inbound isolation per workspace possible (instead of tenant‑wide all‑or‑nothing).  Outbound Access Protection (OAP) lets you block all outbound calls from Spark—and now extends to Warehouse and SQL analytics endpoints—with managed private endpoints for explicit allow‑lists. That’s data exfiltration risk addressed at the workspace boundary. 

On the authorization front, OneLake Security (preview) introduces RLS/CLS that apply across engines, including the SQL analytics endpoint when configured for user‑identity enforcement. This is the first credible step toward federated, engine‑agnostic access control in a lake platform. Pair it with Microsoft Purview’s updated governance experience, and the governance plane starts to feel consistent from catalog to policy to enforcement. 

Finally, observability comes to storage: OneLake Diagnostics is GA, streaming access events into your lakehouse so you can answer “who touched what, when and how?” without bolting on a second telemetry stack. 

5) Cost and capacity are now observable features, not footnotes

Fabric’s Capacity Metrics app has matured into the daily instrument panel for many teams, and reservations plus a SKU estimator give finance and engineering a shared language for right‑sizing. The Azure Pricing Calculator does the rest for scenario planning. The direction is clear: FinOps for analytics is part of the product now, not a spreadsheet exercise. 

What to watch at Ignite

Use the next eight days to calibrate your expectations around these signals:

Open tables as a contract – Watch for progress on OneLake’s Table APIs and Iceberg interop, and for guidance on best‑practice partitioning/layout that balances Delta and Iceberg investments.  Vector in the core engines. Look for roadmaps on approximate vector indexes across Azure SQL flavors, and “good patterns” papers that weave Azure SQL, Cosmos DB, and Azure AI Search into a coherent RAG reference. 

Network isolation + policy tooling – Expect deeper demos of OAP, workspace‑level Private Link, and OneLake Security—especially how these interact with Direct Lake models and SQL endpoints. 

Real‑time normalization – With Synapse Data Explorer retired and Eventhouse in place, look for migration accelerators and “design once” guidance that keeps KQL, streaming ingest, and lake storage aligned. 

How to act on this pre‑Ignite

If you’re shaping a 2026‑ready platform, decide on three seams now so Ignite sessions slot into a plan rather than a backlog:

Format strategy: be deliberate about Delta vs. Iceberg—and where each is the source of truth. OneLake’s interop reduces regret, but clarity still wins. 

Vector strategy: pick where vectors live by default—Azure SQL when you need transactional + semantic in one place, Cosmos DB when you need high‑velocity operational semantics, and AI Search when you want pure retrieval at web scale. 

Boundary strategy: standardize on workspace‑level Private Link + OAP patterns and promote OneLake Security from pilot to blueprint as feature coverage expands. 

The bottom line

The data platform has simplified and strengthened at the same time: one lake‑centric plane, real‑time included; vector in the engines you already run; governance in the same pane of glass; FinOps as a product feature; and AI agents meeting governed data in the middle. Ignite will likely refine these arcs, not reverse them. Walk in with a point of view, and use the announcements to tighten the plan you already started. 

Unknown's avatar

Author: Jason Miles

A solution-focused developer, engineer, and data specialist focusing on diverse industries. He has led data products and citizen data initiatives for almost twenty years and is an expert in enabling organizations to turn data into insight, and then into action. He holds MS in Analytics from Texas A&M, DAMA CDMP Master, and INFORMS CAP-Expert credentials.

Discover more from EduDataSci - Educating the world about data and leadership

Subscribe now to keep reading and get access to the full archive.

Continue reading