Post by Sebastian Raschka on X
Sebastian Raschka@rasbt
XAn updated back-of-the-envelope calculation of LLM pretraining costs based on the just-released DeepSeek-v3 report.
And that doesn't even account for hyperparameter tuning, failed runs, or personnel costs.
It really makes me appreciate the value of openly shared model weights!

317 likes6 repliesPosted Dec 26, 2024