Heuristic Detectors vs LLM Judges: What We Learned Analyzing 7,000 Agent Traces

We compared heuristic failure detectors against LLM-as-judge on 7,212 agent traces. Heuristics scored 60.1% on TRAIL at $0 cost vs 11% for the best LLM.