Gemini 3.1 Pro and the Downfall of Benchmarks: Welcome to the Vibe Era of AI
Do we have a new best AI model, or do we have the downfall of benchmarks in general, as a way of capturing machine intelligence? Full breakdown of Gemini 3.1 Pro, guest-starring the new Sonnet 4.6, plus analysis from 7 papers/posts that will give you much needed context. Oh, and a new record on Simple Bench!
https://epoch.ai/ai-explained-datacenters
Check out my fast-growing (!) app, free to use, and code INSIDER15 for Pro: https://lmcouncil.ai
AI Insiders ($9!): https://www.patreon.com/AIExplained
Chapters:
00:00 - Introduction
00:30 - Post-training Dominance
04:00 - ARC-AGI 2 Caveat
05…
Watch on YouTube ↗
(saves to browser)
Chapters (10)
Introduction
0:30
Post-training Dominance
4:00
ARC-AGI 2 Caveat
5:54
Simple Bench Record
8:22
Hallucination Caveat
10:05
Model Card
11:12
Exponential Coming
12:20
Amodei on Generalizing
15:10
One True Benchmark?
17:02
Other Metrics…
DeepCamp AI