Post by Shital Shah on X
Shital Shah@sytelus
XIf you were to train GPT-4, 1.8T params model,
On A100, it will take 25k A100s and take 3-5 months.
On H100, it will take 8k GPUs and take ~3 months.
On B100, it will take 2k GPUs and take ~ 3 months.
- Jenson at GTC.
819 likes15 repliesPosted Mar 18, 2024