Post by Matt Shumer on X
Matt Shumer@mattshumer_
XThere's a new king of open-source.
The new LLM from @databricks just beat Mixtral!
- 132B total params (16 experts), 36B active (4 experts)
- Trained on *12T* tokens
- 32K context length
From my initial tests, I'm really impressed.

456 likes8 repliesPosted Mar 27, 2024