Live from Transition-AI 2026: Inside Google’s massive AI capex

Training is what drove data centers to gigawatt scale, and inference does not need that. Vahdat says you could serve on much less than a gigawatt, probably less than 100 megawatts, though not arbitrarily small, because an accelerator needs co-located compute, storage and networking around it. There is also a lifecycle effect: whoever is training wants the newest chips every year, so after a year or two a gigawatt of training capacity is handed down to serving. But he does not follow this to the conclusion people expect. Google has found it easier to build a smaller number of larger sites, with asterisks for fault tolerance and geographic spread, and a thousand sites each holding a tenth of a percent of capacity carries management overhead of its own. His answer, offered with an explicit apology for imprecision, is a medium number of medium-sized sites augmented by a few large ones. The case for spreading out strengthens later rather than now: today a generated token is slow enough that a user in San Francisco cannot tell whether they are served from the East Coast or Europe, which is not true of search or maps, so the speed of light only starts to bite as models get faster and more interactive.

Reliability is where he wants the argument to land, and he says plainly that the high standard is not intrinsic. The historical logic was that compute made up a small fraction of the cost of a software service, so overprovisioning it was cheap; many Google data centers target four nines of availability, with all the uninterruptible power supplies and backup generators that implies. Constraint has inverted that arithmetic, because compute is now a much larger share of service cost. Put the trade to an internal customer as four nines and half the capacity against two nines and twice the capacity, and very often, though he is careful to say not always, they take the capacity. Two nines means 3.65 days of downtime a year, which he does not minimize; the bet is that the other 51 and a half weeks at double capacity are worth it. Asked whether this is actually happening, he says only that it is.

That is what makes grid flexibility possible rather than merely virtuous. In March, Google reached agreements with utilities for a gigawatt of demand response across its fleet: during the utility’s single highest-demand week it will brown down, letting the utility plan for something below its worst week and lowering cost for Google and for other ratepayers. Note which way this cuts on behind-the-meter power. Vahdat says Google would prefer grid-connected capacity, because going behind the meter means provisioning all that reliability yourself. On-site generation is a different kind of latency play, the time it takes to get capacity delivered, so bridge power covers the gap while Google invests with utilities to bring transmission a year or two later, and could in principle be mobile. Making any of it work depends on microgrid software the industry is underinvested in, because a browndown is not the site going dark. It is a decision about which 20, 30 or 40 percent to shed, which workloads drain and which service commitments move elsewhere. Kann presses on whether Google can do this only because it is unusually vertically integrated, designing its own chips alongside the rack, the building, the power source and the models. Vahdat does not dispute it. He describes that integration as precisely the point, a chain of custom interfaces each worth a few percent that multiply into something meaningful, and does not say how the capability travels to anyone lacking the stack.

Asked to rank the rate limiter between power, chips and labor, he declines, adds data center construction and delivery as a fourth category broader than labor, and says that at 10am it is labor, at noon power and at 2pm chips, every single day. Pushed with a hypothetical $300 billion budget, he still refuses, saying all of them are at the limit of what Google can do and none is inherently easier to scale. The cost question rhymes with the reliability one. A building is designed for a 25-year life across five or six chip generations, and the watts per linear foot of a disk rack and a GPU rack differ enormously, so committing a building to a single purpose buys efficiency at the cost of fungibility. That trade was not worth making when compute was a small share of cost. It is now.