AI infrastructure numbers are hard to find with any precision. There are many reported numbers of “[company] spending Xbn on infrastructure this quarter” and “[company] has bought 100k H100s or “has a cluster of 100k H100s” but when I went looking for an estimate of how much compute a given company had access to, I could not find consistent numbers available. Here I’ve tried to pull together information from a variety of sources to get ballpark estimates of (i) as of EOY 2024, who do we expect to have how much compute? and (ii) how do we expect that to change in 2025? I then spend a little time talking about what that might mean for training compute availability at the main frontier labs. Before going into this, I want to lay out a few caveats:
These numbers are all estimates I’ve made from publicly available data, in [...]
Outline:
(01:36) Nvidia chip production
(02:38) 2024 production
(03:32) Previous production
(04:06) GPU/TPU counts by organisation
(06:00) Estimating H100 equivalent chip counts at year end 2024
(06:06) Microsoft, Meta
(08:03) Google, Amazon
(10:46) XAI
(11:08) 2025 - Blackwell
(13:53) Summary of estimated chip counts
(14:17) Model training notes
The original text contained 13 footnotes which were omitted from this narration.
First published:
November 28th, 2024
Source:
https://www.lesswrong.com/posts/bdQhzQsHjNrQp7cNS/estimates-of-gpu-or-equivalent-resources-of-large-ai-players)
---
Narrated by TYPE III AUDIO).
