ViralLoop

The numbers on these inference GPU benchmarks seem low. Here are the theoretical values from my model of 8xB200 inference for NVLink, 8-bit, and 70B Llama model, which is closer to 300k tokens/s. This assumes perfect implementations (close to what OpenAI/Anthropic has).

157 likes3 repliesPosted Jun 25, 2024

Remix this post

Your note

Private to you, and used when you remix this post.

Viral score

Based on absolute engagement — the platform doesn’t expose follower counts here, so large accounts score higher.

From likes, comments.

Why it worked

Opens with a short, self-contained line that is easy to take in at a glance.
Uses concrete numbers instead of vague claims, which makes it credible and quotable.
The opening line stands alone, so it reads completely before the post is expanded.
Written from experience rather than as advice, which is harder to argue with.
Short enough to read in full without expanding, which removes a step before the point lands.
Short sentences keep the pace up, so the reader never has to re-read a line.

The mechanics

Hook: Plain observation
Structure: Hook → Landing
Style: Conversational paragraphs at a steady pace.
Tone: personal, concrete
Mechanisms: Specificity
Written for: People building with AI

Analysis by ViralLoop — not written by the original author.

Post by Tim Dettmers on X