Steel For Fuel N° 016 of 56 26 Jul 2024
AI versus computing efficiency
by Andy Lubershane, Partner and Head of Research, Energy Impact Partners
In this note
The question
Is computing hardware getting efficient fast enough to offset how fast AI is adding electricity demand?
The answer
No, and not by a close margin. Efficiency gained about fourfold over four years while training compute grew about a thousandfold.
03The argument
The post is the first of an intended “Charts of the Week” series, and it argues by setting two charts against each other rather than at length. The first is the good news, and Lubershane is emphatic about it: the energy efficiency of leading-edge graphics processors improved nearly fourfold between 2020 and 2024, a gain he says is unmatched anywhere in the economy since the LED bulb, and one the industry has sustained for over a decade. The second chart is the problem. Over the same four years, the compute needed to train the leading generative models rose by roughly three orders of magnitude. Against a thousandfold rise in demand, a fourfold efficiency gain does not bend the curve, and that gap is his explanation for why data centers abruptly became the electricity sector’s dominant subject. What he leaves open is which of the two trends breaks first.
04What you need to know first
- Orders of magnitude
- Factors of ten; three of them is a thousandfold increase, which is the figure doing the work here.
05Details worth keeping
- The efficiency chart is adapted from Thomas Mann at The Register; the compute and hardware charts are from Epoch AI, which Lubershane calls the best data-driven analysis on AI he knows of.
- He offers data centers dominating that year’s Edison Electric Institute conference, keynoted by Jensen Huang, as evidence of the shift.
06Claims worth citing
All figures as stated on 2024-07-26, and read off charts rather than derived in the text.
- Leading-edge graphics processor energy efficiency improved nearly fourfold from 2020 to 2024. Lubershane, from The Register
- Training compute for leading generative models rose about three orders of magnitude from 2020 to 2024. Lubershane, from Epoch AI
07Where it’s contested
Lubershane gives his own position as a bet, not a finding, and names the other side: he expects investment to come back down to earth rather than demand growing in further orders of magnitude, and says smart people he respects expect the opposite. Nothing here tests either view, and both charts are reproduced rather than interrogated.