The real talk on agent evaluation

Key Takeaways

Welcome everyone to real terms for AI on the road. We're here today with Chris Overhalt, developer advocate for Google Cloud. Chris, this is all about agents, but for software engineers. Yes. First thing, what's that aha moment when you're starting to build agents? Yes. That makes it relatively relatable for software engineers in that. That's a great question. So, you got to pick the right thing. Pick two or three components, not one. It's too simple, not 10. Pick two or three. Okay? Give your agent a good model like Gemini 2.5 Pro. It's great. Uh give it some good tools like rounding. And throw in something there like stories, long-term stories. You do that, you're going to come out, you're going to win with two or three, no more. And with that, you'll have your own proof of concept that like, hey, I can build my own system. And then you add another one, another one. So, my hello world was an agent with long-term memory that can Google stuff because that's what I do for a living. Okay. So, I got one. Yes. So, that was awesome. Let's make it a little simpler. I like simple. What's an agent? An agent is a program. I'm I'm You said software engineer. We You said software engineer. So, I say it's a program with job. So, okay. Program. We're in the same universe. Yeah. Okay. Realistically, it's a program. It probably has loops. It probably has control flow, things like that. That's I'll leave it at that. I don't want to get into goals and all this. It's program that has loops. Let's take it a different direction. Okay. Share your thoughts on vibe coding. You know, I've never been at a This is the first time at a conference where I vibe coded the content for the conference the day of. And we're So, here's the thing, but we're not just vibe coding, right? We have different segments. We have web devs, we have Python devs, we have vibe coders. So when I put that hat on and I hit 2.5 pro, here's what I will say. Get a strong backend. Just any it takes one request and vibe code your front end. Problem solved. Hey, you beautiful agent. I'm doing that. Agent in action. I'm doing that with Vue. I'm learning Vue on demand. There you go. Yeah. All right. So last question here for Chris. Yes. I saw something about evaluation. So let's talk about aensic evaluation. That's deep. Uh how how would we break this down? Right? Because it it feels massive. It is like a big problem, but I know that we have ways to do this. Where's the first place to start? Explain like five. Oh yeah, I got you. Evaluation. How well is my model performing over time? Okay. Or is this change good or bad? Right? And so you can look at it of what is your metric? What are you measuring? Doesn't have to be perfect. Just pick something, please. And then as soon as you connect that feed loop feedback loop back to your agent, you'll be smooth sailing. As long as that's disconnected, you have no feedback loop. You're sort of like just five agent, right? So when you talk about agent evaluation, remember that all the layers underneath are model, reasoning, tools. So you can't evaluate an agent without evaluating those layers. I think that's the takeaway, right? And we have ways to do all this stuff. We'll be posting links as well in the YouTube channel for it. More we learn, the easier we make, so you can just put those oneliners to production. Will you give us a happy prompting? Yes. Happy agenting. Oh, you're making it fancy. All right. Happy prompting on three. Yes. One, two, three. Happy prompting. Heat. Heat. N. [Music]