AI is Already Building AI — Google DeepMind’s Mostafa Dehghani

The MAD Podcast with Matt Turck · Beginner ·📄 Research Papers Explained ·1mo ago
Are we truly on the verge of AI automating its own research and development? In this deep-dive episode of the MAD Podcast, Matt Turck sits down with Mostafa Dehghani, a pioneering AI researcher at Google DeepMind whose work on Universal Transformers and Vision Transformers (ViT) helped lay the groundwork for today's frontier models. Moving past the hype, Mostafa breaks down the actual mechanics of "thinking in loops" and Recursive Self-Improvement (RSI). He explores the critical bottlenecks holding back true AGI—from evaluation limits and formal verification to the brutal math of long-horizon reliability. Mostafa and Matt also discuss the shift from pre-training to post-training, how Gemini's Nano Banana 2 processes pixels and text simultaneously, and why the "frozen" nature of today's models means Continual Learning is the next massive frontier for enterprise AI and data pipelines. Mostafa Dehghani LinkedIn - https://www.linkedin.com/in/dehghani-mostafa X/Twitter - https://x.com/m__dehghani Google DeepMind Website - https://deepmind.google X/Twitter - https://x.com/GoogleDeepMind Matt Turck (Managing Director) Blog - https://mattturck.com LinkedIn - https://www.linkedin.com/in/turck/ X/Twitter - https://x.com/mattturck FirstMark Website - https://firstmark.com X/Twitter - https://x.com/FirstMarkCap Listen on: Spotify - https://open.spotify.com/show/7yLATDSaFvgJG80ACcRJtq Apple - https://podcasts.apple.com/us/podcast/the-mad-podcast-with-matt-turck/id1686238724 00:00 Intro 01:17 What “loops” in AI actually mean 05:04 Self-improvement as the next chapter of machine learning 07:32 Are Karpathy’s autoresearch agents an early form of AI self-improvement? 08:56 AI building AI: how close are we? 10:02 The biggest bottlenecks: evals, automation, and long horizons 12:36 Can formal verification unlock recursive self-improvement? 14:06 What is model collapse? 15:33 Generalization vs specialization in AI 18:04 What is a specialized model today? 20:57 Could top AI rese
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related AI Lessons

⚡
The ABCs of reading medical research and review papers these days
Learn to critically evaluate medical research papers by accepting nothing at face value, believing no one blindly, and checking everything
Medium · LLM
 ⚡
#1 DevLog Meta-research: I Got Tired of Tab Chaos While Reading Research Papers.
Learn to manage research paper tabs efficiently and apply meta-research techniques to improve productivity
Dev.to AI
 ⚡
How to Set Up a Karpathy-Style Wiki for Your Research Field
Learn to set up a Karpathy-style wiki for your research field to organize and share knowledge effectively
Medium · AI
 ⚡
The Non-Optimality of Scientific Knowledge: Path Dependence, Lock-In, and The Local Minimum Trap
Scientific knowledge may be stuck in a local minimum, hindering optimal progress, and understanding this concept is crucial for advancing research
ArXiv cs.AI

Chapters (11)

Intro
1:17 What “loops” in AI actually mean
5:04 Self-improvement as the next chapter of machine learning
7:32 Are Karpathy’s autoresearch agents an early form of AI self-improvement?
8:56 AI building AI: how close are we?
10:02 The biggest bottlenecks: evals, automation, and long horizons
12:36 Can formal verification unlock recursive self-improvement?
14:06 What is model collapse?
15:33 Generalization vs specialization in AI
18:04 What is a specialized model today?
20:57 Could top AI rese
Up next
Microsoft Research Forum | Season 2, Episode 4
Microsoft Research
Watch →