Tim Dettmers@Tim_DettmersReading the report, this is such clean engineering under resource constraints. The DeepSeek team directly engineered solutions to known problems under hardware constraints. All of this looks so elegant -- no fancy "academic" solutions, just pure, solid engineering. Respect 👏Opens with an observation
Tim Dettmers@Tim_DettmersMany people think AI will continue improve towards AGI. In my new blog post, I argue that we will not reach AGI due to physical reasons. Key items discussed: The physical reality of computation Why GPUs will no longer improve Why superintelligence is a fantasyOpens with an observation
Tim Dettmers@Tim_DettmersBeating DeepSeek-V3 with a 405B Llama base is not easy -- solid post-training goes a long way. The nice thing is that it is fully open-source, so anyone can use this recipe for their base models.Opens with a number
Tim Dettmers@Tim_DettmersThe numbers on these inference GPU benchmarks seem low. Here are the theoretical values from my model of 8xB200 inference for NVLink, 8-bit, and 70B Llama model, which is closer to 300k tokens/s. This assumes perfect implementations (close to what OpenAI/Anthropic has).Opens with an observation