Tool Calling with LangChain

Key Takeaways

hi everyone Chester from Lang chain here today to show you some features we are releasing around tool calling that we're very excited about so tool calling is uh an increasingly popular feature um supported by a swath of uh of language model providers including anthropic coher Google mrr open AI uh and and others um so the uh basic idea um is that when you send a request to a language model you can supply um some tools that the model is aware of and if the model feels that uh it could uh answer your question by using the tools then it will respond with uh an invocation of of the tools um so what I want to show you today uh and that we are releasing are three things so one is um kind of standards for how we pass uh uh tools to models um two is uh how we work with uh calls of tools from models and the third is a uh a way to create agents um that work with these new s sort of tool calling features um okay so we can get started uh I will enable tracing in Langs Smith uh just by setting this environment variable um so for passing tools to llms um not much new here actually so um we uh support a few different ways of defining tools um you can use functions uh you can sort of decorate them with this tool decorator here I'm defining two tools one is to add integers um and the other is is to uh multiply integers so I have tools add and multiply um you can also use pantic uh so here we will um Define sort of a pantic class and this just tells the model what it needs to add multiply integers it needs two integers um so we have tools there um not too much new here uh so the um uh thing we are releasing is just binds tools across a few different model providers so this has uh been supported for open Ai and a few others for a while but we're just standardizing on it so anthropic supports it um open Ai and actually a uh swath of others um so coh here fireworks mrol vertex grock um all will support uh bind tools so this is how we pass uh tools to llms um so not too much that's new here um one thing that is new is is what happens when you call tools uh so let's try this with uh anthropic first so we're going to use uh Claude um which is a pretty big model so it takes a bit but we will ask it what is 3 * 12 also what is 11 + 49 um and we'll see what it comes up with okay so we have a AI message here so this is not new um but what is new is this uh sort of tool calls um attribute at the end of the methods uh message so we can look at this um so and what we have is uh two these are typed dicts so we have name arcs which is a parse dictionary of of arguments um and uh and an ID and some providers provide this ID so if you look at the content for anthropic um they do things in their own way so they have uh you know a text block that sort of indicates uh you know the reasoning that the model's uh doing and then it it has uh these uh blocks which are of type tool use with a name input um ID um so looks pretty reasonable um open Ai and other providers do things differently so uh here we'll ask the same question of GPT 4 uh turbo so here we have an AI message as usual um but the tool calls are are sort of in their own parameter up here so we have function uh arguments you know with a Json string for the arguments name and and we have this type uh which which is which is function um but now we have them formatted in the exact same way with these uh typed dicts so same thing name args you can see that these are are now equivalent um this just makes it easier to work with tools across model providers and not really have to think about the specifics of of how each provider represents um uh this one thing um one additional thing is so sometimes providers will use uh uh strings to represent arguments like this is a Json string if there's any ever any issues um parsing those arguments they'll show up in in valid tool calls there's none this time um so this just lets you uh expect sort of valid tool calls in in this tool calls attribute um so that's how we work with tool calls uh one additional feature is that we support streaming here so I put in sort of an artificial uh a delay um but you can watch the sort of chunks of the tool call stream out and you can see we have these partially formed uh tool calls which which is kind of cool these accumulate onto in this case I'm accumulating onto one message chunk so so kind of chunks of a message will have string arguments um representing their tool calls and they will also parse them into into valid tool calls um so we've set this up to work in a streaming context the last thing I wanted to show you is uh agents so we have a new uh Constructor for agents called create tool calling agent so any model that uses this new uh tool calling Paradigm um can work with this function so we've had this before for open Ai and uh models that work similarly to open Ai and how they structure their tools um uh could could uh work with our sort of agent Constructors um but now uh any model we think that uh uh uh supports tool calling we can use here so um it works sort of in in the similar way to what we've had uh previously so you have a prompt here we'll just use mraw why not um Mr all large uh I Define two functions here so magic function um which uh applies a magic function to an input uh here it just adds two to to the input um but the model doesn't know what it does and a get word length which will just return the length of the word in in characters so we have these two functions we create tool calling agents we pass it the model tools and the prompt and put it in agent executor um so that works the normal way and now let's ask this what is the value of magic function 3 also what is the length of the word crysanthemum and so mrra we see uh invokes the two functions it gets uh five for the first answer and and 13 for the second answer and it says the value of magic function 3 is five and the length of the word crysanthemum is 13 um so this appears to work uh last thing I will show you is uh that this will um work with with Langs Smith so um we have this call to the agent EX exor here you can trace through all the steps you can see the individual tool calls um and the the call to the llm with the the invocations of of the tools and the um and the final output down here um okay so that's it for today um excited to see what you build with this we are eager for your feedback on it um and as always uh welcome your your contributions to uh the codebase as well thanks