Yam Peleg@YampelegHeard a leak from one of the frontier labs (not oai tbh), they reached an unexpected HUGE wall of diminishing returns trying to brute-force better results by training longer & using more and more data.. (more severe than what is published publicly)Opens with an observation
Yam Peleg@Yampelegso I think it is safe to assume that all major players have reached the limits of training longer and collecting more data already.. It is all about data quality now.. which takes time..Opens with an observation
Yam Peleg@Yampelegimagine dropping 40T tokens into 2T parameters with 10M context length multimodality and then just uploading it to huggingfaceOpens with a number
Yam Peleg@YampelegChatGPT pro coming? - Unlimited access to o1, o1-mini, GPT-4o - Unlimited advanced voice - Access to "o1-pro mode" - $200 per month source: t.coOpens with a question