The mechanics of data center flexibility

The setup is a mismatch that looks like free money. When a hyperscaler asks to connect a 400-megawatt data center, the grid operator studies the worst case: not just whether it can serve 400 megawatts across all 8,760 hours of a year, but whether it could do so in the worst hour of any year over the next seven to ten, with a transmission line out and record air conditioning demand. The facility does not behave that way. It ramps over years as halls get filled, may run at three-quarters utilization even during an intensive training run, and produces a jagged profile: training runs spike by tens or hundreds of megawatts, dip at synchronized checkpoints, then drop away when the run ends. Inference is smoother but still hard to predict. So there is obvious headroom between what is reserved and what is used.

The move that makes the episode worth reading is that Sivaram refuses to call the grid wrong. He says the risk-averse study method is neither irresponsible nor analytically incorrect, because a customer granted a 400-megawatt entitlement may well use all of it, and some large data centers do run flat out. The headroom is therefore not free. It becomes usable only if somebody can convert “probably will not draw full power” into a commitment the operator can plan against, which is a different problem from the one the load profile suggests.

Delivering that commitment is where the mechanics come in. The physical route, running backup generation, is usually blocked by air permits. The computational route works on the load itself: pause or delay a job that can tolerate it, such as fine-tuning a model for an enterprise customer that does not mind if it finishes an hour later; slow a job down; change how many chips are allocated to it; or go down to the silicon and cut the clock frequency so computation happens more slowly and draws less power. Flexibility can also be spatial, moving a workload to another site. What makes this hard is the dual optimization. The grid needs a precise reduction, not a megawatt over the agreed cap, while every workload has its own tolerance for delay, throttling or power capping, so the system has to know what it is running. Sivaram is direct that this is far harder than pausing a batch of jobs.

The evidence and its limits sit together. In Phoenix, Emerald AI worked with the Electric Power Research Institute, Nvidia, Salt River Project and an Oracle data center to hold a 25% reduction for the three hours the Arizona grid needed, using ensembles of workloads specified by Databricks’ chief AI scientist, who reported that only about 10% of a representative cluster’s workloads were strictly non-preemptible. The 25% target was not arbitrary: it is the threshold from Tyler Norris’s Duke University paper, which argued roughly 100 gigawatts of spare capacity exists on US grids if large loads can curtail about that much for around two hours, up to 200 hours a year. What remains unsettled is the contract. Two decades of service-level agreements promise essentially uninterrupted power, and flexibility needs a third product between a guaranteed instance and a preemptible spot instance: an almost-firm agreement where a customer is left alone 99% of the time and gracefully power-capped for the rest. Sivaram’s evidence that this will sell is conversations with hundreds of AI companies plus the logic of scarcity, since customers who cannot get compute at all have reason to trade a small bounded performance risk for more of it. His answer on whether the new contract works is “probably yes.” Grid operators need to see it before they believe it, which is why his pitch to a room of utility chief executives was to join an escalating series of demonstrations.