Fuzzing in the GenAI Era — Leonard Tang, Haize Labs

"Evaluation" is one of those concepts that every AI practitioner vaguely knows is important, but few practitioners truly understand. Is "eval" the dataset for measuring the quality of your AI system? Is "eval" the measure, the metric of quality? Is "eval" the process of human annotation and scoring? Or is "eval" a third-party dataset run once to benchmark a model? To mitigate this cacophony, this talk will provide an opinionated and principled perspective for what we actually mean when we say “evaluation”, beyond the traditional for-loop-over-a-static dataset. In particular, this perspective draws heavy inspiration from *fuzzing*, i.e. bombarding AI with simulated, unexpected user inputs to uncover corner cases at scale. This factors into sub-problems regarding: - Quality Metric. What is the actual criteria we, as humans, are using to determine if an AI system is producing good or bad responses? How do we elicit these criteria before the human SME can articulate them? How do we, as efficiently as possible, operationalize this criteria with an automated *Judge*? - Stimuli Generation. Given a metric, how do we know, with confidence, that an AI system is performing well with respect to the metric? What data is representative and sufficient for discovering all potential bugs of an AI system? And how do we generate this complex, diverse, faithful data at scale? We will discuss in detail the philosophy, technology, and case studies behind both problems of Quality Metric and Stimuli Generation, and how they interact in concert. Timestamps 00:00 Introduction to Haizing 01:16 The "Last Mile Problem" in AI 02:47 The Brittleness of GenAI Applications 03:54 Examples of Brittle Chatbots 04:29 Inadequacy of Standard Evaluation Methods 06:09 Haizing: Simulating the Last Mile 08:43 Scaling Evaluation with Agents as Judges 09:29 Verdict: Accuracy vs. Latency 11:47 Scaling Evaluation with RL-Tuned Judges 14:06 Fuzzing vs. Adversarial Testing in AI 14:37 Simulation as Prompt Opt