Post by Aran Komatsuzaki on X
Aran Komatsuzaki@arankomatsuzaki
XGoogle presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
1B model that was fine-tuned on up to 5K sequence length passkey instances solves the 1M length problem
t.co

1.1K likes26 repliesPosted Apr 11, 2024