Agent Analytics powered by BigQuery

Key Takeaways

How are users engaging with your agents? And what are these agents actually doing behind the scenes? In this video, we'll show you how the new BigQuery agent analytics plug-in for ADK tackles agent observability head on. By adding just a few lines of code to your ADK application, you can capture trace data about your agents execution and stream it directly to BigQuery. This plug-in captures events like user input or the LLM response as well as the agent's tool usage. Having this data in BigQuery allows us to run advanced AI analytics. So, let's head to BigQuery's new agent hub to do just that. We'll kick off a new conversation with an agent that I've built. And it's super simple to build your own agent. You just select some tables as your data source and provide some highle system instructions. And from here, I can ask my agent to run a cost analysis on my agents for the month of November. This agent is using the conversational analytics API under the hood to interact with our data and surface the answer for us. The agent analytics plug-in captures token usage for each sub aent in our ADK application. And once you have that, you can infer the cost associated with each agent. And we can dive deeper to inspect things like the tool latency or the success rate of our tools. And these insights can be used to identify bottlenecks or errors and improve the agents logic and execution. And beyond metrics, these logs also capture trace data like the user input and the response provided by the agent. This allows us to analyze the sentiment of our users and keep an eye on how that sentiment changes over time. A huge advantage of having these logs in BigQuery is that it's very easy to take our logs table and generate embeddings. Having embeddings will allow us to perform a vector search or a semantic search over this data. So instead of trying to search and find a particular keyword, we can instead provide a concept. So I will ask what questions have users been asking about thermalware. Behind the scenes, this will run a vector search in BigQuery that will understand the intent behind my question and provide responses that are semantically similar to my query, even if they don't contain the exact keyword thermalware. So, this is really powerful. With just a few lines of code, you can capture a wealth of trace data about your agents execution and store it in BigQuery for real-time analysis. Check out our code lab for this feature if you want to get hands-on right away.