Post by Samuel L Smith on X
Samuel L Smith@SamuelMLSmith
XAnnouncing RecurrentGemma!
t.co
- A 2B model with open weights based on Griffin
- Replaces transformer with mix of gated linear recurrences and local attention
- Competitive with Gemma-2B on downstream evals
- Higher throughput when sampling long sequences

271 likes9 repliesPosted Apr 9, 2024