Why AI Agents Shouldn't Replace Your Fraud Models

Varant Zanoyan, Co-founder & CEO of Zipline AI and original author of Chronon — the open-source feature platform built at Airbnb that now powers Stripe's charge path, OpenAI's Sora 2 personalization, and Netflix content ranking — explains why AI agents should NOT make high-stakes decisions directly, and what to do instead. This talk introduces "agentic experimentation": a pattern where agents iterate on production ML systems (creating features, training new model versions, deploying to dev) while a human reviews and ships — without ever touching live infrastructure. Varant breaks down the three challenges that kill most agent-on-prod-ML projects: infrastructure sprawl, safety, and reproducibility, and shows how branch-based isolation + semantic hashing + compute reuse make it actually work. Topics covered: - Why fraud detection, search ranking, and underwriting CAN'T tolerate full agentic decisioning - The difference between agents replacing models vs. agents improving models - How Chronon went from Airbnb payments fraud to powering Stripe, OpenAI Sora 2, Netflix, Uber, and Roku - Branch-based resource isolation: keeping agent experiments off production compute - Partial aggregate caching and compute reuse so agents don't blow up your infra bill - Semantic hashing for reproducible agent-generated pipelines - Data isolation without losing cross-team feature sharing - Resource limits as the real organizational guardrail when running 2,000+ experiments - Why agent-written SQL across Spark, Flink, Kafka, and Airflow is unreviewable - The handoff: what an agent should produce so a human can actually ship it to prod For ML engineers, data platform teams, and anyone building agentic systems on top of business-critical pipelines. Links and Resources: - Zipline AI: https://zipline.ai/ - Chronon (open source): https://github.com/airbnb/chronon - Chronon docs: https://chronon.ai/ - Varant Zanoyan on LinkedIn: https://www.linkedin.com/in/vzanoyan/ - Zipline AI $7M seed anno