Is debugging an AI system fundamentally different than debugging a traditional software system?

Key Takeaways

Is debugging an AI system fundamentally different than debugging a traditional software system? And the answer is unequivocally yes. And there's just a a core difference >> [music] >> between an AI system, where I interpret AI as being generative AI, transformer based, >> [music] >> probabilistic in nature from traditional software system, which is deterministic. A line of code does the same thing and it stands any concurrency issues or non-determinism like >> [music] >> uh the timestamp is going to behave the same. So, traditional software debugging is let me run it again, let me see what happens, let me fix it, and and try and restart it and see if that fixed the problem. [music] With generative AI systems, where at the core of it you've got decision making that is based off of uh large language model, every time you run it, you might get a slightly different answer. In fact, you might get drastically different answers because they're inherently probabilistic. So, what you're going to see is the LLM goes off in a certain direction, >> [music] >> and it might uh today say one thing, and tomorrow say another thing. And these kinds [music] of non-deterministic trajectories compound each other. So, now when you talk about agentic systems, you have one LL AI or LLM making one decision, calling some tool, and then the response to that tool causes another LLM or the same LLM to then produce another answer and another decision. And as you build these decisions on top of one another, you could actually have two trajectories that diverge very widely uh from run to run. >> [music] >> So, how do you debug those kinds of systems? How do you control them? One of the things to do is uh to actually run them in production multiple times and see what the behavior is going to look like and put guardrails and constraints around those cases where the AI might be going a different direction than you want it to go. And [music] fundamentally, that's the there's no uh silver bullet to the non-determinism. In fact, that's one of the beauties of uh generative AI is it's non-deterministic. >> [music] >> but in some yeah, fundamentally different types of systems.