Why Most AI POCs Fail and What It Actually Takes to Get Agents into Production l Dev in the Details
AI agents are easy to demo — getting them into production is the hard part.
In this episode of Dev in the Details, Dev sits down with Ben Scharfstein, former Head of Product for Enterprise at Scale AI and now building Applied Compute, to unpack why most AI proof-of-concepts stall — and what actually works in the real world.
This conversation goes deep on:
* Why AI POCs fail to reach production (and why it’s not the models)
* The real blockers: governance, risk, data access, and org friction
* What “forward-deployed engineering” actually means (and what it’s not)
* How enterprises get comfortab…
Watch on YouTube ↗
(saves to browser)
Chapters (16)
Intro: Why AI agents struggle in production
1:35
Ben’s background: Google, startups, Scale AI, Applied Compute
3:20
The early days of generative AI and startup pivots
6:10
Why “everyone wants LLMs” but no one has use cases
8:45
Why enterprise trust takes years (Cursor, Devon, Cognition)
11:30
Scale AI’s enterprise pivot: from data to agents
14:15
Why most AI POCs never make it to production
17:40
Governance, risk, and why enterprises avoid high-ROI use cases
21:30
Why AI POCs are different from software POCs
24:50
Context engineering vs prompt engineering
28:10
Forward-deployed engineers explained (and why it’s not consulting)
32:40
Why enterprise deployments move so slowly
36:10
How execs unblock AI adoption (real retail example)
40:00
Guardrails, red teaming, and AI safety in practice
43:30
Small models governing large models
47:20
Why Applied Compute focuses on “spec
DeepCamp AI