Natural Language Interfaces to Software with GPT-4o Function Calling

Key Takeaways

hi there data scamps and data champs this is Richie now large language models get a lot of hype and a lot of attention for being able to help you write text so whether that's marketing copy or your data analysis code or that Lord of the Rings fanfiction novel you've been working on however there is another use case that is perhaps even more important and that's adding a chat interface to other software it's the basis for how AI agents work but it's also great for making any other software easier to use So today we're going to step through uh a workflow for how you create this sort of uh natural language chat interface to software and we're going to use a data analysis example so uh today's guest is one of the greatest living data scientists yes it's me uh sorry there a little bit egotistical but I hope some of you recognize me by now we've done a lot of sessions together so uh yeah great to have you all here with that I am going to uh go to some slides to begin with alrighty so what are we doing today first up uh the data we're going to use is sales of crossons from a Korean bakery back in 2019 so it's fairly small data set and uh I apologize to anyone who's friends here I did include Pano shakala in the data set I know they're not technically a crosson but too delicious not to include in the data set uh so main goal for the session is just to figure out when should you make use of function calling capabilities uh from GPT or any other large language model and how do you go about doing that and then we're also going to get into a little bit about the architecture of applications that use generative Ai and why are we doing this well frankly pretty much every software company is frantically trying to add generative AI features to their own software products this this is possibly the hottest skill on the planet right now there's like so much demand for this very important thing to learn now uh the motivation for this session actually kicked off um when I interviewed Rob Wilson on data frame so uh if you're interested in this please do go back and listen to this episode uh that is a hyper Link in the slides if you want to go to it I'm sure Reese will also post a link in the chat but basically he started off this little rant about how uh Siri Alexa Google home like all these uh AI assistants they've been around for like a decade now and the way they work you speak to them and it's just kind of a thin layer of translation into some deterministic software underneath so um I think the terminology they tend to use is like uh is it Electa has like skills so basically it's other bits of like traditional software underneath and the AI layer is relatively thin it's just deciding which one of these uh bits of uh um bits of traditional software to use now there's a lot of hype recently about oh yeah we're going to make an application that's entirely a generative Ai and that's really stupid uh we know what to do this sort of thing has worked for a decade and it's still the right approach you just want a little bit of generative Ai and then lots of traditional software and so it's basically just a natural language interace to uh existing software uh so go back and listen to that episode here's the kind flow of his thesis generative AI is just very very expensive still compared to traditional software I mean like a few cents for uh running a query that's a lot compared to you know the basically next to nothing for uh performing calculations uh traditionally and you already have a lot of traditional software that works very well so you should do as little as possible generative just do let it do what it's good at which is providing a chat interface so anytime you want to computation don't try and do that with GPT or your other large language models try and do it with something that's good at calculation and so uh this function calling feature that we're going to cover today this provides the bridge between these two pieces of software now the flow is slightly tricky you have to call GPT twice to get an answer to your question so uh this is what we're going to cover if you only remember one concept from today this is the this is the part you need to remember so you start off you ask a question your first prompt is just saying please can you uh solve this problem for me can can you uh do whatever uh usually like in this case we're going to be saying can you answer a question but you could also say can you generate some text and whatever and you're going to tell it as well as just asking the question in the prompt you can say these are the functions that you are allowed to use now GPT is going to then respond it's going to say uh these are the functions that I'll need to use in order to answer your question and these are the arguments I'm going to need to pass to it so once you have that information you then calculate uh you call those functions you do the calculations locally like not in the AI on your own machine uh or in your own cloud and then you collect the answers and then you pass uh those the details of those function calls with the answers back to gbt and then the second time it's got uh your original question it's got the details of the functions to use and the answers it's got everything it needs so it can now answer your question so there's this backwards and forwards flow all right uh enough slides let's get coding here we go so uh before you set up uh please do make sure that you've got access to uh the open AI API so you need to uh sign up for a developer account there um if you have a look in uh the getting started notebook it's going to give you details on how to sign up uh the other thing you need to do is make sure that you have uh the API key connected so you need to go to environment environment variables and in this case uh I'm not going to show you where my my variables but you you're going to want to add a new envir variable call it uh open uh AI API key e past in the value from your developer account uh give it a name click create and then you need to make sure that you are connected in this case um you can see a green tick next to it so uh you're going need to have access to the API in order to do anything today uh all right so I'm going to go back to the notebook and um the just go while while we've got this open so notebook is is the one you're going to f notebook solution is got the answers uh and bakery sales is the data set we're going to use uh the rest of the files in here you don't really need all right so uh We've covered this uh with Rob Wilson and before you get started you need your developer account and your API key we're going to make use of a recent version of the open AI uh python package so uh run that to install uh the latest version the syntax changed a few months ago so um you in fact that they're updating this package uh every few days so you need to make sure you have the latest version all right we also need to make use of a few packages so we're going to import uh the pandis package uh usual Alias PD we're going to import the Json package Jon and from the uh open uh yeah from the open a package uh we're going to import uh open AI uh this Cas sensitive to make sure I get the capitalization right and then also from uh the I python. display package we need uh two functions uh called display and markdown that's just going to help us have prettier output so we run that that's my inability to type shouldn't copy to paste that so um okay so uh we've got pandis Json uh some stuff from the open AI package and some display functions now uh just to test this before before we make use of the function calling interface let's ask a question that uh GPT has no chance of answering so uh first thing we need to do is uh set up uh an AI client model and to do this you just call open AI the open AI function you don't need any arguments to it and it's additional to call this variable client and next we're going to create a prompt now um with the open AI package prompts contain or any sorts of messages are dictionaries with two elements so the first element is a role so user means it's um a message that you've created so that means a prompt and then content is the text that you actually want to tell it um and oh and then you're going to have a list of these dictionaries is like your full conversation or the full chat so in this case we're going to ask it uh did I sell more almond crons at my bakery in August or September now it doesn't chat obviously knows nothing about your bakery or how much you sold at your bakery uh so it's not going to be able to give you a good answer but we're goingon to ask it anyway so now uh we're going to uh cre at completion basically means we're going to send it a prompt we're going to get a response um I've got some code hints here um so uh we're going to say response is equal to I think we just copy and paste all this directly so client is so this is the the model we're using so this is uh GPT uh we we're performing some chat functionality we're asking it to complete some text and we're creating so this is the perhaps the most common function you're going to use when you're working with uh the uh the the chat interface to GPT uh you can pick a model you can actually probably get away with um GPT 3.5 here if you um if you if you're feeling a bit uh strapped for cash GPT 3.5 is a bit cheaper to run um I haven't tried whether whether or not it works but we're not doing anything complicated so you can possibly get away with it all right so uh here we are uh sending our chat this is our conversation to GPT and we're asking for a response all right so here are the results you get a chat completion object back and there a ton of information there the most exciting bit is the uh the content which we're going to pull out now um okay so um we actually want all of this message to begin with so um there's a load of just like uh metadata Gump that we don't really care about but then there's messages is the the important bit and within the message there is content so I've just shown uh what the code is here so we're going to get the response message it's if you only have if you only ask for like one response which is the default then it's always going to be in the response. choices zero because there's only one of them and then message and then we're going to pull out the content from that and then display it as marked down this is kind of it's boiler plate code which is why I'm not making you fill this in it's the same sort of each time and this is the response we get so I don't have access to your bakery's um records or sales data um but this is what you have to do in order to find out so it's trying to be helpful it's trying to give you some information about like what you can do in order to find this out but obviously it doesn't have the data so it can't give you an answer this is how uh we're going to uh help it so uh ignoring GPT or any AI for a while we're next going to figure out how do we calculate the answer using python um all right so we've got a data set um the sorry the full data set is on uh kaggle I have trimmed it down to just data in 2019 and I just included uh crossant related data rather than like all the different um different products the bakery was selling um and so oh it also gives you like individual timestamp salale so I've just aggregated the day level so we've got a small data set I think I can't remember how how big the thing was but it it's it's literally uh a few kilobytes and we got three different columns uh so date time is date of the sale product what was sold we got the four well three types of crosson and Pano Shaka and then the number of products they sold that day so uh let's take a look at this data set uh rather than me just talking um about it to you so we're going to call it sales and we can use pd. read CSV now one useful thing because I can never type file names correctly L if we go back to show workbook files we go to Bakery sales and the dot dot dot and you do copy path to clipboard and then I'm going to paste it in here and that needs to be quoted um so we're going to read it in and it's also useful to print out some column info so we can use sales. info that's a functions we need parentheses uh let's run this ah right okay so uh we've got three columns here uh we've got um the date time prod n all now this is um this it's just read the dates into the string so actually we need to do one little fix here and uh trying to remember how you pass date times in um uh in Panda so we're going to use past dates equals date time we need to give it the name of the column to pass as a date and square brackets and we're going to run this again and now you can see if we look at the column info uh this datetime column is now a datetime object so it's treating it as dates rather than just text and here you can see we've got information we got um this is each day so it goes August through to the end of the year uh we just looking at crons can maybe um let's skip through to the end you can see every on you got uh and then we got alond crons and then numberers sold for each day so we got 608 rows pretty three columns pretty small data set but it's enough to uh have some fun playing around now um because the question we asked was how many uh was oh what was the question we asked so it was like did I sell more almond crossons at my bakery in August or September um we need some code to uh answer this sort of thing so I provided a bit of code for you so you can see what this does it takes the sales data set query does uh uh filtering uh rows of of a data frame and then this is an example so we're looking at alond Cross s in uh September we're getting the N sold column and then we're calculating the sum so that was 25 now next step because we're talking about function calling we need to be able to write a function so we're going to wrap this sort of code into a function and generalize it so next oh there's some code hints if you're struggling with uh the pattern for how you uh how you write a function the idea is we Define a function and we give it the name get total sales and this is going to take two arguments product and the name of the month we need a colon at the end there and if we enter so that's indented four lines what I'm going to do I'm going to cheat a little I'm going to copy this and paste it into the function definition uh those need indenting ones so this is going to get calculate the number sold we call that nld and we don't want it to always be calculating alond crossant so we're going to change this to product and it has to be at product this is um a special Syntax for the pandas query function if you passing variables in there you need to prefix them with an at and the same thing is we don't want the month name to always be September we want this to be at monore name so this is calculating the number sold and the next thing we want to do is we want to return the number sold but uh because GPT is a large language model it's designed to work with text it uh it expects the return values from the functions to be string so we're going to convert this to string I'm just going to run the cell to Define it so uh let's try running this just to have an example so uh okay we're doing Pano in October just to see if this works okay and we sold 80 Pano Shaka in October that seems that seems plausible Al it's returned this number as a string which is uh just what we want all right so step three uh we're gonna um speak to GPT um this time we're going to tell it about the existence of the get total sales um function now we have to describe the tools that are available to uh to gbd um now this is what happens when um software developers come up with ideas for code um because a lot or a lot of um like the standard way of accessing this sort of stuff is with JavaScript JavaScript developers like really love their like nested objects you know like in the same way that web pages um HTML is just sort of nested tags everywhere JavaScript is all just like arrays and dictionaries just nested inside each other they're like very complex uh objects and so uh when you're defining the functions that are available it's this sort of annoying nested syntax not very data analysis like um you know I spend too much time in pandas I'm I'm used to rectangular data but uh this is the way uh software developers like to do things so I've given you the template we're going to fill in some details uh there's a bit more information about like what's going on in the codent but for now uh let's just fill in uh fill in the blanks okay so first of all the tools um you can have many tools so as well as functions you can also have things like um file search and um code interpreter capabilities but so you have many tools we've got a list on the outside and then each individual tool is dictionary so we've got the the curly braces it's a function so it has Type function and then function details again it's a dictionary the name of the function is get total sales we're going to paste that in there we need to provide a description of what the function does this is and you can't skimp here because this is how um GPT is going to know whether or not it's going to want to call the function so you got to have a good accurate description there and then parameters that's going to describe all the arguments of the functions this always has Type object and uh properties are the names of each of the arguments that go into the function so uh the first one is uh product uh so product is a string we need to describe what it can do um okay so let's provide this definition it's just the name of the bakery product now in this case we only have four uh data for four Bakery products available so we want to restrict the values that um that uh that this uh product can take and you use that uh it's called enum so enum short for enumeration it's another name for a categorical variable so yeah if you come from a software development background you're more likely to say enumeration rather than categorical variable but it's the same thing all right so uh we can have four possible values here we can have four possible values uh so CR on Al CR CR on Pano Shaka and so I'm going to paste this in to the enum uh element of product and then I'm going to do similar sort of thing for the second um variable so the second argument so the second argument is called month name again this can take a string value and then argument two uh so the description is going to be well this is the month that we're going to consider sales for all right so again I mean because this can only take the values August September October November December we could make it um an enum as well just say it's only allowed to to take one of these values GPT at least GPT 40 is smart enough to know that these values are months so just saying saying it's a month you don't need to give it any extra advice uh the other thing is that um both these uh both these uh arguments both product and month name are needed in order to make the um get total sales function work so we can specify these as required arguments so let me just fill this in here so we've got product and then we've got month name all right so I have to say I struggle a bit with like the these very nested objects it's very easy to have like a typo somewhere and then everything's a complete mess but hopefully this is going to work there we go and now um all we have to do is call uh GPT uh again in a similar way to what we did before so I'm going to go back and copy and paste code because we know that code worked before so we're just going to see how to edit it so we're going to go all the way back to task one let's copy this so we're copying the client. chat. completion. create code and then we're going to do a little modification all right so uh the modification we're going to make is going to say tools the tools argument and it's just we call this variable tools so we're going to say tools equals tools and just need to change the uh the variable name I'm going to call this response one so because we're going to call G GPT twice where like we're going to call the First Response response one second response response two all right so we have a uh completion um and then we got to extract the message from the response um so I showed you the code last time um you need to basically the same thing this time I'm going to give you 30 seconds to uh get started to see if you can figure out what to write and then I'm going to um then I'm going to show you the answer all right let's go for this so we're going to start with uh response one and then we're going to have a look at the choices and we want the first one of these and then we're going to get the message element so let's just run that and see if that worked that looks uh about right so we're going to call this uh response message and then I'm going to display the contents of this uh display down let's run this okay and SS there was uh we've got an object but there's absolutely no content in there at all um yeah so this is completely blank um actually we don't even need this we can just maybe do the whole the thing as is so yeah completely empty um this is all this is this is correct it's is it's what we want um so the next thing to look at it did return something so rather than considering the content this time we're going to look at the tool calls element and this is this is the useful bit here from that first response so it tells us which functions it thought be appropriate to call now we only gave it one option which was the get total sales function uh and as you can see so it it it's a list with the square brackets and it's as a call to make two calls so you've got two two two instances of this chat completion message tool call so the first time uh it wants to call get total sales setting product to aled quason and month name to August and then second is same again with September this makes sense because we asked it did we sell more Elman crons in August or September so all this is very promising now then we need to extract um the sort of the useful bits from this from these tool calls and then go and run the code get the answers collect all the information back up to send it to GPT again uh before we do that though we need to add uh the response message to the end of the chat so we've got our chat and we're going to call append and we're going to append response message all right so now um we've got two um we've got two uh tool calls to deal with so normally you have to write this code in a loop we're going to do that a bit later but I think it's just clearer if we just step through considering um the first tool call and you can see the flow uh individually all right so uh we've got tool call is equal to that response message Dot tool calls and we'll just take the first one uh maybe I'll just print this as well uh for now just so you can see what's going on yep so this is our first um the first tool call now we need to get the name of the function to call so that is hidden within the uh function uh. name element so function name is is tool call. function. name and let me uh show you this this time so it's get total sales that works and then we need to get the the function to call from that name so uh this is the name of the function just the string we want the actual get total sales function now there's a trick here I've I've described it in the code hints because it is a bit um tricky but uh there's a few ways of doing this the easiest thing is that globals will give you a list of all the variables that are available um actually let let me just put this in a in another cell for now just you can see what it looks like so it gives you a dictionary of all the different um functions that are available and so you can find variables by name so if you say globals and then get total sales that is going to return the get total sales function so this is exactly what we want here still WR that and then we also need to get the function arguments and again this is um a slightly tricky uh thing here so if you look at tool calls and then actually I can't remember where it was let me just run this again so it's going to be tool calls. function. arguments just following the the structure of of this uh thing so this is giving us a dictionary which is what we want but it's also inside a string which is frustrating so we actually have to call json. loads and because this is It's a python dictionary string but it's also a Json dictionary string unfortunately uh we can use this to convert it into a real dictionary so this is now given us a python dictionary um and let's just call this um Mars um and so now we actually have to call the function uh so we're going to say function to call and instead of passing individual arguments when you call a function you can pass a dictionary of arguments you just need to pref prefix it with double asteris uh so let's uh let's call this and you'll see how this work works so that's giving us the answer so this is calling um get total sales and it's passing product was I can't remember what was on onqu on and month name is August and it's giving us this answer so now we just need to edit this to say function response equals that I'm going to run it one more time and now we need to collect all these details together so the we need an ID for this um and if you have so let me just create another um thing so you can see what was in tool core so you see within tool core there was this ID element so every time you call a function uh gbt creates this um ID code for it so we're going to add tool call. ID the name of the function we stored as function name so that goes here and then content is the return value from the function that was the uh function response so this is all the details that we wanted uh and this is the form that uh GPT needs for when we call it the next time all right so with did that once if you have many function calls it'd be rubbish to have to do that every single time so you actually need uh a for loop I have uh written a function that sort of condenses all that so we can just when we do the for Loop we can just uh call this function so you're going to have to run this to define the function it's just it's exactly the same stuff as as these last few blocks let me delete that again so we're can do this in a for Loop so we're going to say for Tool call in RSP oh it's called RSP message instead of RSP message one there's a slight difference in uh variable names there dot tool calls so we're going to Loop over those and so then we're going to call this function which was get function call details all right so you just need to make sure that uh this argument you're passing in matches whatever you call this at the start of the for Loop so four tool call in in all in all the tool calls the function call details are equal to get function call details when you're passing tool call and then we need to append the results to the chat so we're can to use chat. append again and we're going to pend those function called details and now after this we're going to have a look at the final state of the chat all right so this is our conversation with GPT so far um I would say just be careful to only run this chaten stuff once if you run this cell multiple times you're going to end up with extra message in there it might make things weird but anyway uh so first message was uh to gbt that was uh did I sell more alond crossons we got the message back that basically said these the functions I'll need to call to answer it then you got a message saying these are the details of my first function call another message another message saying these are the details of my second function call all right so last step then is called GPT again including all the function call details so uh I am going to go back and copy and paste uh this uh from whenever we last called GPT all right so this time response two and we don't need to tell it about the tools this time all we have to do because it it's it's got all the details all we have to do is just uh give it the chat and it's got everything it needs in there all right so we run this and now it's got some information in there so we're going run this last line to pull out the content and there we go it's now going an answer saying you sold more onqu in August compared to September it's given us the numbers even though it didn't have access to our data set it's just got it just had the computed answers it has managed to solve the problem so effectively what we've done here is that gbt has given us this chat interface to uh the data analysis all right um that's it uh for the code long so if you're interested in learning more about this we have got a whole skill track on developing AI applications I heartly recommend that you take that like it's such such a hot skill at the moment that uh yeah you're not going to regret uh learning more about it and then I also did um code along last month month before on creating AI assistance um there is um a function calling interface for the AI assistant um I was like it it was going to be too long to try and introduce that today but that's the kind of the next thing you want to learn about is trying to I would say go through like learn how to create an a assistant and then try and apply the stuff you just learned about this function calling to that assistant uh that's how you get to creating your own AI agents all right so uh any questions please so we're a little bit light on questions today but I think that just means it was uh an incredibly comprehensive Cod along so if anybody's got any questions now then uh yeah now's your chance but I think that might be it for us today yeah I mean uh we have got plenty of time for questions so if you do want to ask me anything um I know some of that was a little bit involved like there was some you know they got those nasty like um nested objects for uh content and like the flow having to call GPD twice it gets a bit confusing so um it's a slightly more advanced topic but if you could follow at least half of this then I think you're well on your way to becoming AI development Legend that's how you make make the big books and uh retire early all right so uh if we have no questions in that case uh I will give you your time back we do have another AI related webinar happening uh I think Tuesday is the next one uh so please uh do come back for that uh we're be looking at fine-tuning open source llms it's going to be focused on mistol uh we've got a couple of uh dat Camp code along regulars and Andrea and Joseph uh doing that and then next Wednesday we've got a session on the data Camp Q3 road maps you're going to find out what's coming next for data camp and Thursday there's a session on Bridging the generative AI Talent Gap so if you don't know anything anything about generative AI or more importantly your colleagues don't know anything about generative Ai and you have to have stupid conversation with them you find out how to upscale them all right brilliant uh I hope you've enjoyed yourself and please