AI Running Its Own Code: Agentic Code Interpreter

Key Takeaways

hello Mike here now one thing that we've talked about before with large language models is that they're not very good at mathematics if it's a super simple maths question then maybe you're in luck but if you're asking something more complex especially with a large data set an analytical problem that you want to have solved that's not what large language models are going to be able to do for you but one thing they can do is write code so if we can take code that the large language model writes and then run it on behalf of the large language model and provide that data back again now we're on track to be able to answer these more complex questions so in this video I'm going to show you what's behind this here's a simple chatbot which I've created and if I ask it a question such as plot a graph of a Fibonacci Sequence actually a large language model may have seen that kind of thing before so let's say plot a graph of an inverse Fibonacci Sequence and let's say uh minus um 123 I don't know something that is unlikely to have been plotted before now if I send that into this particular agent then after a short while and a little magic edit it comes back with this exactly what I wanted this graph so how did it manage to do this well it wrote some python code we ran it for it in a sandboxed secure environment and it brought back what we wanted let me show you how to do this so if we go to Amazon Bedrock here I've got an agent that I've started to build because it's an agentic workflow that we're going to use for this so here is my demo agent because no imagination and inside of here I've selected the Claude 3 sonnet model to be the model behind the agent and I've given it some instructions now we'll look at these instructions in a tiny bit more detail later but they're Fair extensive talking about how to solve mathematical problems and how to be Analytical in your thinking using a tool to run python code on your behalf now I haven't enabled that yet but I get to do that by clicking on additional settings and going to enable for code interpreter it's in public preview as I'm recording this video and it connects a python execution sandbox environment to this agent so it can perform well EX ly what we just saw happen so if I go back up to the top here click save then my agent's ready to go now I could show you this working in the playground environment inside the console what I really want to do is show you how to do this programmatically so you can build it into your own application so here I have a Jupiter notebook with all of the code that you'll need to get a sample of this up and running and yes a link to this is in the description beneath this video so let's jump right into into it let's load some python libraries I'm including map plot lib here because I want to be able to plot the graphs out to the screen as we're watching it here if if we have graphs um and we've also got boto 3 of course which is the SDK for AWS for python so let's skip through this we're going to Define ourselves a region that we're going to work in now this needs to be a region that supports Bedrock it needs to support agents and you need to have the correct models enabled inside of your account as well and this particular one we're going to be using the sonnet model from anthropic so with the region selected let's go and create our agent now this next cell of code here is huge there's lots of things going on so let's quickly step through it and then I'll press play essentially what it does is it creates an agent it then um adds to it the code interpreter and it makes it ready for us to be able to play around with all in one go so we're going to use use the Bedrock agent um endpoint here so the client for our boto 3 is the Bedrock agent there's two of these there's Bedrock agent there's Bedrock Bedrock agent runtime so the Bedrock agent one is the one that we need for configuring the service the Bedrock agent runtime one is the one that we use to work with the agent once we've created it so I'm going to use some python code here to create the agent I'm also going to create some IM am resources because we need to create a policy for agent to use so I'm going to define a name that I want to use for this um and then here is our instruction so this is the instruction that actually had in the console just a moment ago so you're an advanced AI agent with capabilities in code execution and on it goes now I actually used anthropic clawed model itself to help me write this system prompt because that's essentially what this is this is a large part of what essentially will become the system prompt um which just talks through how to do problem solving data analysis doesn't give it code examples but gives it a sense of how its personality can be used to create the solutions to the problems that we have so that's a big long prompt look through it in your own time um then we are going to Define which model we want to use and we want to use the sonnet model um we're going to get a random suffix such that we can create Resources with unique names and then we get into the creation of stuff um so here we're going to create the IM am policy so it's got a trust policy so that this is going to allow Bedrock itself to call a Bedrock model in this case it's going to be the Sonet model in our region that we're working in so we're going to call um I am create role to make that and put role policy to create a role with an attached policy once we've got that then we can get on with creating the actual agent itself now to do that we're going to to call the create agent function on top of the Bedrock agent we're going to pass in basically all of those parameters that we defined just above so the agent name the model that we're going to use the instruction and the role um and then we need to wait for it to become not prepared so the next state after preparing or or creating is for it to get into this not prepared state so we're just wait for that and then we're going to attach an action group now action groups inside of agents are where the tools live and so if you're making your own Action Group you can have your own tools that go and connect to whatever you want this is a special one this is a special one for code interpreter so we're going to create an action group we're going to call it code interpreter action we're going to enable it we're going to attach it to the draft version because at this point we don't have any other versions of our agent but this is what's really important here this parent action group signature value this value of amazon. code interpreter will convert or make this Action Group into our code interpreter Action Group that's all this will be and it's all it ever needs to be it's a special if you like templated action group for executing code so if you've already got an agent and you've already got your own action groups you can create another one with this method and create our code interpreter that's it that's all we have to do we don't have to do anything more than that this is managed for us by the service so once we've created our Action Group we're going to get an action group ID note that before we actually got an agent ID so we've got our agent ID came back and now we've got our Action Group ID we're going to then um wait for our Action Group to become enabled that's the state that we want it to be in and then we can prepare our agent so this is the equivalent of clicking the prepare button in the console if you've done that before um we wait for it to become prepared and that's basically just the status of it um and then we get to create our Alias Now with an agent once it's created in order to be able to actually access it from code we need the agent ID and we also need an agent Alias ID and this Alias links to a version so basically with this we're going to have one version one Alias but the Alias gives you the ability to be able to do Release Control so you can have different versions running on different aliases so you've got some control over which clients using which versions um but in this particular case super simple one version one Alias ID so once that's fully prepared then we get to the end of all of this code with an agent ID and an agent Alias ID that's all we need to be able to work with our agent so let's run this cell it's going to zip through this it should take no more than 30 seconds or so to create all of that for us um right here inside of the notebook now you can do this as well with things like cloud formation but the support by by cloud formation for the things that have just come out in public preview such as code interpreter might not be there yet so we're doing it in Python plus it runs inside of this notebook so there you go how long did that take 21 seconds 22 seconds and we're all done okay so next up I'm going to go and grab myself a Bedrock agent runtime client object the runtime object for me to be able to um invoke our agent so let's run that and have a look now this is a big function I've got here just for the purposes of us testing this um but you get to see how the different pieces of code work in here um for our particular use case so I've created an invoke function and it's going to basically wrap around in voke agent but it's also going to get the response back from that and print it out to screen so it's be really just a a play Swiss armun knife function that I've created um so we're going to invoke the agent we're going to pass in our question in input text there um and we're going to use the agent Alias ID agent ID there all of the particular variables and requirements are just listed out here on the right hand side now if we do want to pass in some of our own data into the agent then we can do so and you do that here with session State um and follow this syntax to be able to maybe send in for example a data set so you can absolutely pass data in as well as receiving data back out as well so let's um have a look what we do here so once we get the response back from invoking the model we get the completion and we get our event Stream So the event stream is a stream of output that we get from this model now now agents don't stream text but what they do do is they stream some debug information they will send you a chunk which is all of the text which is the output but all in one go um and it also will send you files in this particular instance as well so we need to handle that and that's what this code will do so we're going to handle the chunks um of which there will only be one with any given generation and so that'll be the text output and then we've got the files and the file outputs now there could be more than one file so we have to Loop through everything that we've got and we basically grab out the name which is passed back from the environment the type so the mime type of the file and the actual data itself now in this particular code if the data I get back is a PNG or a of type PNG then we're just going to we're going to render that into the notebook here if it's anything else then we'll save it to the disk because we don't really know what to do with it but we can get access to it and we can look at it there so that's what this function does um a bit of a catch all does everything kind of function just to help us test the code interpreter functionality so let's um scroll down a little bit and let's set ourselves up with a brand new session so I'm going to create a session ID and we're going to go ahead and invoke our model so calculate the sum of the Fibonacci sequence up to 1,000 so if I click play on that and you'll be able to see in seconds here how long this is currently taking in this public preview of code interpreter but what's happening behind the scenes here and if we were actually to look at the trace then we could actually see um the question going in it being mixed with our system prompt there that then going off to create a Sandbox environment and all these things happening well it's already come back 16 and 1 half seconds later the sum of the Fibonacci sequence up to 1,000 is 2583 and it's figured that out by running some code not by trying to figure it out using language um I can say plot the Fibonacci sequence so this is very similar to the graph that we pulled back uh a little bit earlier and so we'll see that obviously it's going to bring back the file and wow it's massive but it's brought back that graph for us and a bit of text I've generated the plot of the Fibonacci sequence so that's it bringing back a file using that section of code that we had which was right here okay and well let's do something more complex here so I'm going to restart the session here um and look at something which is maybe a little bit more typical of the kinds of things you might want this to do plotting the Fibonacci sequence not that useful in the grand scheme of things but this may well be so look we've got here A bunch of text information with some numeric and data information in here about some billing data so and what we're going to ask it to do is use a billing data provided below to create a bar graph of this data so categorize it add it all up make a graph and then get some insights from this data so again this is not the kind of thing that we could ask a large language model to do normally because it really wouldn't be able to read and understand this it's certainly wouldn't be able to draw an accurate graph but if we run this it will pass this data in um and hopefully I'll produce this graph for us and we'll see that in just a moment now this is the kind of data that you might have gotten from a previous step in an agentic workflow so maybe um maybe an agent has using say for example the clae 3 model which is great at extracting text from images maybe it's grabbed an image from the web somewhere it's seen something it's substracted the numeric information here and now we're passing it into this subsequent step where we're going to produce some output like this now super big super super super big but you get to see that it has produced the graph and it makes sense you can see total spend on this axis here and then clothing Electronics category and home down the bottom there and if we look inside of the table of data if you could call it a table of data that aligns with what we're seeing in there as well so it's had this language understanding of the data but then used the execution environment to actually properly analyze the data and beneath it here well we got um some analysis which is actually much wider than the screen is um much some analysis of what that data means as far as it is concerned so super useful and you can start to see how powerful this is to have a code interpreter combined with your large language model so the last thing I want to do here is I also want to show you how we can um get this data out potentially so I'm carrying on the conversation here with the agent I'm saying format the sales data into ajacent format and save the file so let's press play on that and because the type of this data won't be a PNG it won't try to render it inside the notebook but it should save the data out to a Json file and it says it has so if we go and have a look in our files there it is and there is all of our data from that initial prompt saved out to ad Json structure which means that I can now use that in a subsequent step that I might have which is my own code potentially as well okay hopefully that was interesting hopefully that was useful that's how to connect a code interpreter environment up to an agent um without having to manage with your own sandbox environment and all of the security that comes with that this is fully managed inside of agents for Amazon Bedrock this is in public preview now so if you've got any feedback about this at all then please do put it in the comments below uh and put any questions down there as well that you might have about this if you liked this video then please do consider giving it a thumbs up and subscribe to this channel again for more videos just like this check out this video which I've got which is about another public preview um uh capability of Agents which is long-term memory and there's many more videos yet to come so until next time I'll see you later