Bob McGrew@bobmcgrewaiThat o1 is better than GPT-4.5 on most problems tells us that pre-training isn't the optimal place to spend compute in 2025. There's a lot of low-hanging fruit in reasoning still. But pre-training isn't dead, it's just waiting for reasoning to catch up to log-linear returns.Opens with an observation
Bob McGrew@bobmcgrewai“Models keep getting more impressive at the rate the short timelines people predict, but more useful at the rate the long timelines people predict.” Good post.Opens with an observation