How to build context systems for AI agents

Key Takeaways

[music] Welcome back to Real Terms for AI with me, Aza, and Jason. And Jason, we've been talking a lot about how to design agents and how to build agents and agents and agents and agents. I want to talk about something different today. Do you have any ideas? >> I was thinking we should talk about systems for agents. >> Not sure that many agents is actually that different than talking about one agent, but sure. But you got to promise me you're not just going to lob a whole bunch of technical terms at us and that you're actually going to like use the board and draw a picture so that we can all follow along with what's going on in your head. Deal? >> Maybe. Well, remains to be seen. >> Okay, >> let's talk about system. So, to do this, I think we should go back to the pet shop agent that we first talked about, right? >> Okay. >> And let's go ahead and get this drawing going because I need to draw it out. So, we have our order agent here. >> Okay. And this is for our pet shop. So, say buy cat food or you maybe place an order for a new toy. >> Yeah. >> And if you also remember, we talked about memory >> that our agent has. >> And we talked about different kinds of memory. We had working memory for the current task or the current prop the agent was working with. We had short-term memory for some of the things that the agent had done recently. And then we had long-term memory, which we talked about as being used like across user sessions. Right. >> Exactly. >> Okay. Now, let's say we want to add another agent, a different type of agent. So, maybe we've taken the time and we have a customer service agent. >> Okay, that agent has memory. >> It has connectors for memory, but we need to get these memories for that agent also >> because this agent needs to know about my orders. >> Exactly. >> So, how do we get these two agents to share memory? So in the application development world, what we're almost talking about is just managing state over long periods of time and things that we know. And this is where we start to think about systems that are helping our agents out. >> Okay. So this memory system supports our agents. >> It does. >> And then each of our agents can connect to that and pull the information that they need for short-term or long-term memory. And we also may have agents adding to those memories. So for example, our service agent if it has a tool call failure, it may go through and add memories about the right thing that it should do in the future or correcting maybe a bad tool call that it had in the past. >> Okay, but this memory can't exist in like just a markdown file or an XML file or anything if we're going to share it across systems because each of these systems needs different memories to do their jobs. >> We could do that. That would probably be the very simple way. But coming back again to application development principles, we could use a database and we could store those memories in a database and we could even have different pieces of code which help us with things like harvesting the most important information from those memories or even you know creating different types of memories that we can share with our different agents. So each agent is responsible for using this memory system that we've created to get the types of memories it needs out of our data store, whatever format that is, >> when it needs to run them and then writing back memory information that other agents or it may need at a later date. >> Exactly. And we could even think about as we add agents. So let's maybe say that we want to create a inventory agent that this agent would also interact with this memory and maybe learn things like you know this one type of cat food is always out of stock and the supplier never delivers it on time. We can use this memory to improve other agents that we may choose to add to the system. So the inventory agent could write that fact in some way to the memory and then the service agent could use that when people call in complaining about not getting their cat food and the order agent could use that information to give a better predicted date of when the cat food might arrive. >> Yep. And it's all powered by a shared system that our agents are using. And in this case it's just our memory system for it. Kind of like a micros service. >> It's a bunch of microservices. Microservices all the way down. >> It is. And if you'd like to learn more about getting started with building your own systems, like a memory system for your agents, we've put some links down in the comments below that you can get to. >> And with that, this is Aza and Jason signing off for Real Terms for AI. >> Happy prompting. [music]