Fabric Can Finally Monitor Itself: Building a Real-Time Capacity Control Loop

Most Fabric capacity incidents are diagnosed in reverse.

A report slows down. A refresh misses its window. A user sees a capacity-limit error. Then someone opens the Capacity Metrics app and works backward through the evidence to find out what happened.

That is useful monitoring, but it is still an autopsy.

Capacity Overview Events change the sequence. Microsoft made them generally available in this month, giving Fabric a near-real-time stream of each capacity’s smoothed utilization and throttling posture. A Summary event represents a 30-second window for an active capacity, except that all-zero windows are suppressed. State events arrive when status changes. Those signals can flow through Real-Time hub and Eventstream, land in Eventhouse, appear on a Real-Time Dashboard, trigger Fabric Activator, and ultimately drive a guarded scaling action through Azure Resource Manager.

This is also a useful way to reintroduce Real-Time Intelligence. RTI is sometimes described as Fabric’s specialist workload for telemetry, clickstreams, and IoT. Capacity Overview Events show the larger idea. Fabric can use its own event-driven architecture to observe and operate Fabric.

The goal is not “add an alert when utilization reaches 80 percent.” It is an architecture that turns capacity management into a controlled feedback loop.

A Fabric capacity is not a CPU meter

The first mistake in capacity automation is treating a Capacity Summary event like a sample from a server CPU graph.

Fabric does not account for work that way. Workloads can burst above the compute represented by their SKU so that an operation finishes quickly. Fabric then smooths the consumed Capacity Units across future 30-second timepoints. Interactive operations are smoothed across a shorter horizon; background operations are generally spread across 24 hours. The capacity is therefore managing both the work executing now and the compute already committed into its future.

The Summary event exposes that operating model. It includes the SKU and its base CUs, CU milliseconds consumed in the window, billable interactive and background utilization, preview utilization, carryforward added and burned down, total overage, and a workload-level utilization breakdown. More importantly, it exposes three different views of approaching throttling:

SignalWhat it representsWhat happens above 100%
interactiveDelayThresholdPercentageSmoothed utilization committed across the coming 10 minutesNew interactive operations begin receiving submission delays
interactiveRejectionThresholdPercentageSmoothed utilization committed across the coming hourNew interactive operations can be rejected
backgroundRejectionThresholdPercentageSmoothed utilization committed across the coming 24 hoursNew background and interactive work can be rejected

These are not three versions of the same utilization percentage. They are three time horizons over the capacity’s future obligation. That makes them better control signals than a single instantaneous percentage.

Figure 1 — Capacity Overview Events expose Fabric’s smoothed capacity ledger. “Real time” here means timely publication of the state Fabric uses to make throttling decisions, not raw processor telemetry.

The second event type, Microsoft.Fabric.Capacity.State, is deliberately sparse. It reports meaningful changes such as creation, pause, or overload. It does not repeat “healthy” every 30 seconds, and a newly connected state table can remain empty until something changes. Scaling and renaming are not state changes; the new SKU or name appears in subsequent Summary events instead.

That distinction matters when we verify an automated resize. The controller should not wait for a special “scaled” state event. Azure Resource Manager is the source of truth for operation completion; the next emitted Summary event can provide independent proof through capacitySku. On an idle capacity, zero suppression can delay that second confirmation.

Reintroducing Real-Time Intelligence from the inside out

I have previously described Real-Time Intelligence as a tight loop: capture, analyze, visualize, and act. Capacity Overview Events are almost a reference implementation of that sentence.

Real-Time hub makes the capacity an event source. Eventstream becomes the routing and light transformation layer. Eventhouse provides durable, high-resolution history and KQL analysis. A Real-Time Dashboard gives the operations team a live surface. Fabric Activator watches for a threshold crossing and starts the appropriate response. The final action can remain a notification, invoke a Fabric item, or pass the decision to an external controller that updates the Azure capacity resource.

Figure 2 — Keep the monitoring and control plane separate from the production capacities it observes. The event path remains useful even when a monitored workload capacity is under pressure.

For a production estate, I would place the Eventstream, Eventhouse, dashboard, Activator, and decision history in a platform-operations workspace on a separate capacity. That creates a real failure boundary. Putting the alarm, the history, and the mitigation logic on the capacity being overloaded is attractive in a demo and fragile in production.

It is also worth remembering that the observer is not free. Event routing, Eventstream, Eventhouse, and Activator consume Fabric capacity, and their workload families appear in the utilization breakdown. The volume from a handful of 30-second capacity events is modest, but the architectural principle still matters: monitoring is a workload, not background magic.

Eventstream should land every event in Eventhouse and send the operational fields to Activator. Capacity events do not backfill, so begin collecting before the first incident. Because the source wizard scopes a subscription to one capacity, an estate-level implementation creates a source per capacity and unions them into the operating stream.

From there, the action plane should be intentionally narrow. Activator can send email or Teams messages, run a pipeline, notebook, Spark job, copy job, or Fabric User Data Function, and invoke a custom Power Automate action. Microsoft’s current What’s New catalog still lists the Activator-to-User-Data-Function integration as preview, although the newer action guide has no preview banner. A function can validate or enrich the decision and hand it off. For the actual Azure mutation, the conservative enterprise pattern is an Azure Function, Logic App, or Automation runbook using managed identity to call Azure Resource Manager. The controller identity receives a custom Azure role scoped to the target capacity, not a subscription-wide Owner role.

The separate-workspace design also has to be reflected in networking. Workspace outbound-access protection blocks cross-workspace event consumption unless the Real-Time Events connector is allowed in the workspace’s data connection rules. Private-link restrictions can require private connectivity between the consumer and the source workspace. Those are design inputs, not post-deployment troubleshooting notes.

There is also a lifecycle wrinkle. Capacity Overview Events are GA, but as of August 2026 the source-specific Eventstream documentation still says Eventstreams containing this source do not support Git integration or deployment pipelines. Until that documentation is updated, treat the capacity subscription as a governed environment-provisioning step and keep the rest of the solution in source control where supported.

For F SKUs, this is scripted resizing, not a newly introduced native capacity-autoscale switch. The Azure management API accepts a PATCH to the Microsoft.Fabric/capacities resource with the target SKU, and it returns either a completed response or an asynchronous operation to track. Power BI Premium P-SKU autoscale is different. Spark Autoscale Billing is also separate: it moves Spark work to serverless pay-as-you-go compute; it does not resize the shared F capacity.

The scaling policy matters more than the trigger

It is easy to build a capacity scaling demo. It is harder to build one that does not flap between SKUs, scale down while carryforward debt remains, repeat the same request every 30 seconds, or spend freely because one workload behaved badly.

The first rule is to scale from Fabric’s pressure signals, not from a single burst. A Summary event contains capacityUnitMs, but a high window can be exactly what bursting and smoothing were designed to absorb. A better rule evaluates a threshold percentage, its recent direction, and its persistence. Microsoft’s own tutorial uses an 80 percent background-rejection threshold as an example—not a universal recommendation—and uses a numeric change condition grouped by capacityId so the rule fires when the value crosses the boundary rather than on every event above it.

The second rule is asymmetric control: scale up quickly enough to protect the experience, but scale down slowly enough to prove the additional headroom is no longer needed. A controller should examine the current SKU, permitted next SKU, recent threshold history, outstanding overage, last action time, cost ceiling, eligible regional SKUs, and maintenance restrictions. Its policy store must map capacityId to the full Azure resource ID because the event lacks the subscription and resource group required by ARM. It should project the smoothed percentage onto a smaller target before scaling down. The operation should be idempotent: if the target is already set or an update is in progress, record that fact and exit. A downsize below a reserved commitment does not reduce the reservation bill, so the FinOps policy needs a reservation floor as well as a technical minimum.

Figure 3 — Hysteresis, cooldown, idempotence, and explicit approval boundaries turn an alert into a control system instead of a resize macro.

The boundary between the F256-and-below family and the F512-and-above family deserves a hard policy gate. Microsoft warns that crossing it in either direction can briefly interrupt the capacity and cancel in-flight work. That is a maintenance-window or human-approval action, not an ordinary automated step.

Pause and resume should be treated with even more care. Pausing can clear throttling, but it makes assigned content unavailable and converts remaining smoothed usage and cumulative overage into a billing event. It is an emergency administrative action, not a clever substitute for scaling.

Capacity overage is another distinct tool. The current preview capability can pay off excess usage that would otherwise cause throttling at three times the pay-as-you-go rate. Its rolling 24-hour limit is evaluated every five minutes, so it is a guardrail rather than a hard cost cap and charges can exceed the configured limit. It preserves availability but does not add memory or improve execution speed. It can buy the controller time; it can also hide a sizing problem at a premium price.

Build ticker tape, not a live monitor

Capacity Summary delivery is best effort. Duplicate events can arrive, occasional windows can be missing, and an active but idle capacity suppresses all-zero windows. A paused capacity emits no Summary events, while pausing can push smoothed usage into the next available window and create an enormous apparent spike. Silence therefore means “unknown,” not headroom. Before an automatic downsize, confirm through ARM that the capacity is active and use an independent workload schedule, watchdog, or approval when missing Summary data makes freshness ambiguous.

Those behaviors are manageable if the raw stream is retained. Microsoft recommends reducing duplicates to one row per capacity and 30-second window, sampling a five-minute maximum to tolerate rare gaps, and excluding utilization above roughly 500 percent from ordinary charts to avoid pause artifacts. Keep those extreme rows in the audit table. Use separate trend and latest-state views so flows, balances, and dynamic workload detail retain their meaning:

let Deduped = materialize(
CapacitySummary
| summarize take_any(*)
by capacityId, windowStartTime, windowEndTime
| extend WindowEnd = todatetime(windowEndTime)
| extend BudgetCUms = todouble(baseCapacityUnits) * 1000.0 * 30.0
| extend UtilizationPct = 100.0 * todouble(capacityUnitMs) / BudgetCUms
);
Deduped
| where UtilizationPct < 500.0 // visualization filter; raw rows remain retained
| summarize
PeakUtilizationPct = max(UtilizationPct),
InteractiveDelayPct = max(interactiveDelayThresholdPercentage),
InteractiveRejectionPct = max(interactiveRejectionThresholdPercentage),
BackgroundRejectionPct = max(backgroundRejectionThresholdPercentage),
PeakOverageDebtCUms = max(overageTotalCapacityUnitMs),
OverageAddedCUms = sum(overageAddCapacityUnitMs),
OverageBurnedDownCUms = sum(overageBurndownCapacityUnitMs)
by capacityId, bin(WindowEnd, 5m)

For current cards and workload mix, query the same rows with summarize arg_max(WindowEnd, capacitySku, overageTotalCapacityUnitMs, capacityUnitUtilizationBreakdown) by capacityId. This takes the latest debt balance instead of summing a stock. Keep historical views time-aware rather than joining today’s state onto past buckets.

The dashboard built on that view should show the three throttling horizons together, current SKU, overage added versus burned down, total carryforward, and workload mix. A separate decision table should record the event window that caused an action, the rule and policy version, prior and requested SKU, controller identity, ARM operation ID, result, approval if required, and the Summary event that confirmed the change.

That audit record is especially important in banking, insurance, wealth management, and payments. A scale-up during morning market activity, a card-processing settlement window, an actuarial model run, or month-end regulatory reporting can be perfectly reasonable. The enterprise still needs to explain why capacity changed, what it cost, and whether the action protected a defined service objective.

What this changes about Fabric usage

Fabric capacity has often been discussed as a box: pick an SKU, assign workspaces, inspect the Capacity Metrics app, and resize when the box proves too small. The box is still real, but the better mental model is a shared ledger of present and future compute obligations.

Capacity Overview Events make that ledger visible while it is changing.

That shifts capacity management from retrospective sizing toward workload operations. Interactive and background work can be governed against different time horizons. Temporary pressure can trigger a notification, a temporary resize, or a deliberately limited overage response. Repeated pressure becomes evidence for permanent right-sizing, workload isolation, or optimization. The same history can show whether a larger SKU solved the problem or merely admitted more inefficient work.

It also changes the role of scheduling. Fabric’s bursting and smoothing already reduce the need to hand-stagger every background job simply to avoid a momentary spike. Capacity events let the team see when the smoothed obligations still stack into a real service risk. Scheduling becomes one policy option alongside optimization, isolation, surge protection, overage, and resizing—not the entire operating model.

The events do not replace the Capacity Metrics app, workspace monitoring, or a tenant-level platform model such as FUAM. Capacity Overview Events provide the fast capacity-level pulse. The Capacity Metrics app provides item and operation detail for diagnosis. Workspace monitoring and job events explain what ran. FUAM connects capacity behavior to inventory, ownership, governance, and longer-term FinOps. A useful platform operations model combines them instead of forcing one monitoring surface to answer every question.

There is one final implication. Real-Time Intelligence is no longer just where Fabric processes someone else’s events. Fabric Events make the platform itself a publisher. Capacity Overview Events tell us that utilization and throttling have changed. Job events tell us that work has started or completed. OneLake events tell us that data has arrived or changed. Business Events can describe what the enterprise believes happened. Once those event types share an operating plane, Fabric begins to look less like a collection of analytics experiences and more like an event-driven platform.

Start collecting the data now, automate later if it makes sense

Most capacity incidents begin with someone noticing that Fabric is slow. Capacity Overview Events let the better conversation begin while pressure is still accumulating across the next ten minutes, hour, or day.

The first move should be simple: subscribe to the Summary and State events and retain them in Eventhouse. There is no historical backfill, so the most valuable baseline is the one already running. Add a Real-Time Dashboard and threshold-crossing notifications next. Only after the organization understands its normal cycles should it enable a guarded scaling controller, with a narrow identity, an explicit SKU band, hysteresis, cooldown, cost limits, and human approval where scaling can interrupt work.

The real importance of Capacity Overview Events is not that Fabric gained another monitoring feed. It is that Fabric can now apply its own real-time pattern to itself: observe the platform as events, preserve the evidence, make a governed decision, take an action, and verify the result.

That is a much better place to begin capacity management than an autopsy.

Unknown's avatar

Author: Jason Miles

A solution-focused developer, engineer, and data specialist focusing on diverse industries. He has led data products and citizen data initiatives for almost twenty years and is an expert in enabling organizations to turn data into insight, and then into action. He holds MS in Analytics from Texas A&M, DAMA CDMP Master, and INFORMS CAP-Expert credentials.

Leave a Reply

Discover more from EduDataSci - Educating the world about data and leadership

Subscribe now to keep reading and get access to the full archive.

Continue reading