Post by Sebastian Raschka on X
Sebastian Raschka@rasbt
XInteresting surprise drop from Thinky! The Inkling model looks pretty solid on benchmarks, and it has some little surprises in its architecture:
- Small conv layers in several places
- An RMSNorm for the embeddings (before the block RMSNorm)
- Rel. position bias instead of RoPE

1.3K likes34 repliesPosted Jul 15, 2026