Working with APIs in Python

Key Takeaways

hi there data scamps and data champs this is Richie I say it's been a couple of weeks since we've done a Cod along I've been getting withdrawal symptoms so yeah uh four sessions this week very exciting stuff now as I'm sure you're all aware there are a ridiculous number of data tools and data sources available uh around the world and on the internet of course so that means that you're going to end up using a lot of them often within the same project and so one of the most important skills you can learn is how to glue things together and build things from the individual components now the standard way to do this is using an application programming interface or API and so today we're going to learn about how to use rest apis for this idea of gluing things together and so uh for those of you who read the session description word for word you may have noticed we planned on building a trading bot turns out uh this is quite hard and if you do it wrong you can lose a load of money so we've had a change of heart and go for a safer option so instead we're going to build and automated daily news feed bot so I promise you this is going to be very fun and pretty useful if you're more interested in finance use cases please do join the session on Friday uh as well now teachings today is one of data Camp's uh more senior uh engineering managers it's Chris ramakers he is the developer platform and design system engineering manager so uh in his work as developer platform manager he's responsible for enhancing the developer experience productivity compliance and security so basically making the other Engineers uh work uh more efficiently uh and more safely and so he also runs the design system team creating tools and components that Empower uh our engineers and designers to develop a good-looking stable and Visually consistent web application so making data Camp look pretty so Chris is also the instructor for the data Camp course introduction to apis in Python hence the uh the title of this session and he also runs the software development company Skyrocket Concepts and he's previously been an engineering manager at several startups so with that uh welcome Chris please take it away thank you for that excellent introduction Richie um hi everybody good morning good afternoon good evening wherever you are um I'm Chris as Rich already mentioned and today we're going to uh be building cool stuff with apis hopefully uh but before we begin I'm going to rehash what re already shared um if you want to follow along with this uh code along you'll need a couple of accounts setup so newsapi.org metos source.com and the openai platform um you'll need an account for all three of these please make sure that you've got your account registered your email address confirmed and if you want to follow along all the way till the end you'll also need to add your credit card to the open AI uh open AI um platform um like res said it won't cost much literally like a couple of pennies um but you'll need it to be able to follow along all the way until the end so what are we going to do basically so first we're going to um collect a couple of um data from different sources basically so we'll first start by fetching news headlines from newsapi.org then we're going to fetch the weather forecast from medios source.com and we're going to feed this along with an AI prompt into the open a open AI chat completions um API once we've basically got a really nice formulated morning update and text form we're then going to feed it into the opening text to speech API which will generate a really nice sounding and realistically sounding morning update so we'll basically have our like personalized forecast of the day along with the news headlines to have alongside their coffee in the morning before we go ahead let's quickly rehash a couple of Core Concepts uh like already mentioned we're going to be working with rest API they are based on the HTTP protocol which is the same as like what the what runs the internet like everything on the internet mostly runs on the HTTP protocol conceptually it's really simple youve got a client sends a request message to a server server evaluates it sends a message back as a response so basically integrating an API is nothing more than you as a client generating a message sending it to the server waiting for the response and then evaluating the response message there's a quick example of an actual raw uh request message at the bottom uh don't be too intimidated about it we'll make things a lot simpler so there's four important concepts related to a apis that we'll need first is URL and URL parameters so a URL is something like an internet address like it's what you type in your address bar in your browser basically um the more important part is like the pieces that it consists of there's three important elements the domain the path and the query or the Ural parameters I refer to these as query parameters or Ural parameters just interchangeably during the code along um but it's important that you distinguish these three uh Main parts of the URLs you'll need them in the codal second is the HTTP verb so your message that you're sending to the website or to the API um we'll have one of four common verbs like there's nine in total but we'll basically mostly use four um get post put and delete get is about getting information back from the server basically like you're sending a get request to the API the API responds with the information that you requested post is about making changes on the server so you creating a new thing on the server put will be updating an existing thing on the server like for example changing the title of a blog post and delete obviously deletes something on the server so keep that in mind the HTTP verb that we'll be using is an important part of our API interactions the third is headers headers is something that you won't necessarily see when you're normally browsing the internet but it's also part of every single message that is sent to a server and back to a to a client it contains con context information on on the message that is being sent so for example if the server that you're talking to the API that you're talking to requires you to authenticate you'll be able to send the authentication information via header and then the last is the status code status codes as part of the response message so the message you get back from the server the response message will contain a status code which will indicate how the server has evaluated your message um it's one of five categories um the most frequently encountered ones are 200 which means everything went fine the messages returned and you can process it uh 404 means like we didn't find whatever you were asking for and 500 means something went wrong on the server so please try again in a later time because something obviously is wrong on on the servers end so that seems all very complex obviously and like if you saw that raw text message uh the message um we don't want to build that manually obviously and that's where like the request library or the request package comes in request makes working with apis a lot easier it abstracts a lot of things away like it it it it makes it a lot simpler to retrieve the status code and the body from a message uh or add headers and request parameters to request messages so without further Ado we can basically dive into the code so in this chat um re should have linked a uh um a message where you can click on and it will make a copy of this exact notebook once you've done that open the uh the outline on the left hand and you go to the code along section the first two parts are basically what we covered in the presentation just a minute ago so the first task here is basically preparing our workbook so in order to basically get started working with apis we need to be able to send our private API keys to um to the API servers obviously um the way that is done in notebooks is not by pasting your API key straight into your python code no we'll need to set up environment variables we'll do that through going through environment and then environment variables and you can click on the add button here to add a new set of environment variables which one will we create there's three the first one is API key metos source second one will be API key news API and the thir one will be API key open AI then we give it a name uh code along API Keys something like that now what are the values for that we need to go to the websites where you just register your accounts so metos source for example for metos Source we'll need to go to the dashboard um if you go here here and our dashboard and you'll see that you'll get to see my API keys on this on screen um I'll re rotate them obviously afterwards um so they can't be um stolen but for the purpose of this code along I'm going to just show them on screen um so this is the API key I can copy it and then I paste it in here same thing for news API I'm going to go to my dashboard going copy the API key that is shown here for news API and I paste it here and for open API you'll need to create a new API key so basically what you do here is you go to your dashboard once you've logged into platform. openen ai.com you go to your dashboard you click on API Keys you do create new API key you give it a name data cam code along for example you give it all permissions and create your secret and then you'll get a one chance basically to copy paste it because otherwise like afterwards you won't be able to get your API key anymore as you can see here it's um it's it's it's um appreciated basically so I have already set up my um API keys but I'm going for the purpose of the code along go ahead here and do test here if you do create it'll take a little while and once that is done we can click next and connect and that will connect these environment variables into your python notebook you'll see a very brief error um that the kernel needs to restart but once that is away your environment variables are loaded and you should be good to go so how do we get those environment variables actually loaded into our python code that's where we come in here let me quickly see I need to be on this notebook all right so once we've set up these environments variables as you can see on the screenshot here that you need to like start importing them into our uh python code so that's where the OS package comes in OS comes bundled with python like it's available everywh you have python 3.x available um so what you do is you just do import OS that's it then what that does is it loads a couple of environment variables into the OS variable like all your environment variables that you have created so the tree that we created but python generates a couple on its own two are now made available to us in Python code so we'll put them in Python variables now so I'm going to create API key Meto Source equals OS Environ there we go so basically what this does is we'll create an python variable called API key metos source and we'll load that environment variable that we created just a moment ago into this python variable we'll do the same for all three of these keys so we can use them later on in our code so this one for metos Source this one for news API and then one from open Ai and if you want to whether it works we can just print one API key metos source for example and see what that gives there we go that's my API key that we just loaded into our python variables and an environment variables you can try the same for all of them but that is all setup that is necessary so if you got these three API Keys set up in your notebook and stored them in these three python variables we're good to go and we can go on move on to the next to the nest ask so we'll start by getting the news headlines first um for that we'll need the news API so news API you can open the documentation here has a couple of API endpoints the first thing that we need to figure out is like how do we authenticate right like how do we identify ourselves with our API key so that we can talk to the news API because if we don't pause in the API key will basically get an error that tells us like you're not allowed to use my API so that tells us the the API um authentication section there's three options here we go and go with the first U we can pass an API key query string parameter so remember what we said earlier in in the slideshow like there's query um query parameters that's where we can add this API key so what we're going to do first is we're going to start by importing the request Library the one that we want to use to like interact with our apis import requests and the nice thing is that the data Camp uh data lab this comes pre- imported like it's pre-installed you don't need to installed anything else if you run this locally you might need to use pip to install requests data Camp uh data lab notebooks come with the request package pre-installed so all you need to do is import it Green Dot says everything went fine the code uh the library is imported now and now we'll create a dictionary this dictionary will contain all of the create parameters that we will sent to the news API so let's call this dictionary news API URL parameters and the one key that we need to add for sure is API key so what we'll do is add API key here and then we'll reference that value that we imported from our environment variables so news API so that means API key news API so now we've got a dictionary which contains one key which is the API key that we need to identify ourselves with the news API but we might need more so looking at the instructions here we know that for example we need the uh top headlines API news API offers many different apis we're interested in the top 10 headlines for today so what does the top headlines API need it needs an API key and from what we can tell nothing else seems required so we should be good to go so that's all that we need in terms of the a the URL or create parameters that we need so now we can go and send that API request to um to the API what do we need as you can see here it is a get request so let's use the get function from the request Library requests.get what do we need to send it to to we copy the whole domain and the part nothing from the Cur parameters because those are already in the other variable so basically what this would do is it would send a request to this all simple get request go fetch everything that is now available on top headlines but we're not passing in our API key yet so we need to do one more step that is passing in extra parameters by doing this we'll ask the request library to add all of the elements of this dictionary as query parameters to this uro so behind the scenes it will add a question mark and then API key is and my API key to this URL but this makes it a lot more convenient to manage your API uh your Cy parameters so this is all that is needed to send this request and obviously we need to capture it in the response so what we'll do now is in the response variable we'll put the output for requests.get and then we're good to go what else do we need to do whether or not to figure out if the API request succeeded um we'll need to look at whether or not the response code is uh the status code uh in the responses is 200 uh like we sh before so what we can do here is response status code equals 200 in that case everything went fine in the case something went wrong it's always nice to Output an error message and in this case we'll raise a system exit something went wrong all right so that should be enough let's give this a try and see what happens oh seems to be an error here I need to add a body to to this part of the if statement let's do print response again an error news API R pram is undefined I need to run this code obviously because otherwise this variable will not register oops something went wrong again so basically this means that we didn't get a 200 from the API back this is kind of expected um because I I've done this before but it's good to basically show you to identify how do you learn what is actually wrong when something goes wrong in the API so what you can do now is you can just instead of planely outputting something right wrong and that's it you can print for example the response text which will give you exactly what the server returned and usually there's contains something like what is wrong with the message so this is what the server returned there's a required parameter please set any of the following parameters and try again sources Q language country or category so it seems like the documentation says that API key is required but it also requires one of these um so what we're going to do we're going to pick country and we're going to send that country along with the rest of the Ural parameters so what we need to do here is then add a country and I'm going to pick the US for the US headlines run this code so that the new variable is defined and then if we run this again we should no longer get an error yeah exactly so we're printing the full response here and as you can see it doesn't tell us much right like it just says like okay there's a response and this 200 indicates that the response has a 200 uh code so how do we get the actual data from the response well that depends but most API that we're talking to will be sending and returning Json um you can usually find that in the documentation all the apas that we're going to be using in this code along will be using Jason so what we need to do here is instead of just printing the response print response and then run the jent function over it so that will basically take everything from that response message and output it as a readable uh python variable and there we have it obviously readable is relative right this is still like structured data and code but there's already a little bit uh more readable and understandable what's in there for example we get a status property back with okay total results 33 um there's articles here which is a list and every item in the list has a source and has an author and has a title and a description and you're all and so on basically so this will allow us to basically like now evaluate that response and extract the data that we need from it so what we're going to do is instead of printing it we're going to put it in a variable so that would be response data for example and then what we're going to do is we're going to create a new list with only the title and the summary um the summary of the description of every headline because that's all we need we're not interested in the URL or in the when it's published like for a morning update all we need is the title and the and the description so what we can do now is um using the technique in Python called this comprehension um to extract that value from from this from the data so what we're going to do is make a new variable headlines headline articles equals and that would be oops that will be a list and in this list we're going to use list comprehension to take everything from the response and Only Store the title in the description so that would mean that there is a new list for every item with title and with description um and then for article and resp response data articles so what this does is it will Loop over the Articles property in the response data and that's this one that we're referring to so like there's a status property a total results and an articles property so we're going to Loop over all articles in the article property and then we're going to create a new dictionary with title and description and drop the article title and the article description in that dictionary so let's see what that does if we then print atline articles and run all right so now we have a list with an itry and obviously like this one is removed apparently the API authors have removed an entry from their database or from their service um we need to deal with that later but it has an entry with only the title and the description for every single news item this is the second one for example NFL scores and then there is another one Giants bench BQ Jones and then there is another one that's also removed and so on and so on so if you want to get rid of the removed ones we can add an extra statement here if article title equals not equals removed so this is basically conditionally going to Loop over all items in this article list and skip the ones where the title is removed so this should give us indeed all of the articles with the title and the description in a response so that's the first step we've gotten the titles from the API the descriptions from the API we've got our news headlines let's move on to the next part I don't know if there's any questions Richie by now um maybe it's a good time to quickly stop and see if there's anything if not we can just proceed sure that was very cool I have to say um when you were getting the errors in the uh URL call I was like do I need to jump in here I was glad you had it all under control it it deliberate it was it was deliberate yeah because in all honesty when you're working with API you counter a lot of errors and it's a useful skill to be able to debug apis that you've not built yourself um absolutely making mistakes and being able to debug them is an incredibly important skill uh we got one question from the audience so far so this comes from M saying you can talk about asynchronous API calls so maybe first of all do you want to explain uh what's an asynchronous API call why might you want to do it and then how do you approach it yeah um well synchronous API calls an async ver API calls are two different things like I guess in Python when you're talking about server to server API calls you won't encounter async API calls that frequently but what the difference is a synchronous API call is basically you send a request and you wait for for the response basically um asynchronous API calls usually are API calls where there's a lot of computing happening on the server where you need to wait a long time for the Computing to be finished so you don't want to wait uh for the API call it could be minutes it could be hours basically like that usually works with a call back for example you fire the request and then you tell them okay when you're done with all your jobs send the response to that piece or that code on my end um it's not at least in application web application development it's not a common thing uh for server to server implementations um I guess when you're talking about more like data heavy big data and data science related apis it might definitely be something um that you'll encounter um it's out of scope for this uh this this codone but I think we might actually have some tutorials on our blog for uh doing asyn calls with with python all right super thank you uh yes so I have to say I've not encountered them very frequently uh but yeah if you're using huge data sets then uh maybe it's um it's a big thing all right so we got one more question just come in I suspect this might be spoilers for the rest of the session if we go through it in too much detail but Maxim asks like how do you chain API calls so the second request depends on the answer of the previous one yeah how do you combine multiple API calls together um well you can chain a lot but then doing that your code becomes pretty unreadable if you ask me um so in JavaScript for example that is something that you quite often see but for example you could go ahead and send like request.get and then the URL here and then do Json immediately and then extract articles and then you could for example do another get request and wrap that in there but like in all honesty I'm head of the developer experience at data Camp I am I'm responsible for a good developer experience and readable code is probably the biggest differentiator between like good developers and um developers that struggle the most I'd say probably like writing readable code is a very important skill in my opinion like code is R 10 times more than it is written so it's important that your code remains readable and easy to digest so I would encourage you to ear towards like not chaining too much and like having your code become very readable think about your yourself like if like two years time you need to dig up your code and you need to ask yourself like what the hell was I doing here then you're probably doing it wrong um think of your future yourself and do yourself a favor and try to keep your code as reable as possible I'm definitely in favor of not writing code that I can't read again a few months later once I forgotten what it does uh brilliant all right uh so uh we'll continue uh Chris you just need to reshare your screen again you might want to talk through the code I just heard you type oh did I stop sharing uh you did stop sharing yeah I think it oh yeah my screen sa kick in hold on um there we go all good all right cool um okay moving on to the second part basically like where we first fetched the news headlines now we're going to fetch the weather forecast pretty similar we're going to do things a couple of things slightly different though um so looking at the um metos Source documentation there's two apis we'll need in this case because if we read the point we and forecast API it tells us that for in order to use this we need to specify either a latitude and longitude or a place ID this place ID is something that we can f use uh we can find by fetching the find places API so before we can get the weather forecast we first need to find the place ID for the location that we're going to fetch the weather forecast for so this will be a 2 API call uh section basically um and in terms of the authentication um similarly to the news API one you could use a uh a c parameter in the URL or in this case an xapi header I generally always prefer xapi uh well X API key headers or whatever headers like adding your secrets to your headers because like if you add them to your query uh parameters they're really really easy to um expose they'll for example be in like any logging that you're doing on the server for example um if you get a choice prefer to add your API authentication to your headers so we'll do just that in this time in this case so let's copy paste this xapi key and we're going to create a dictionary with headers for the request so in this case what we'll need is a key xapi key and then the value for the Meto source API so this is similar to how we for example previously created a u a dictionary with request parameters like you're creating another dictionary with key value pairs one for every header that you want to send along with your API request so in this case we'll be sending the X API key request with the value of this metos Source API key that we have stored in our environment variables let's run this code so that we have the varable for later use if we move on then we need to um use this find places API this find places API is what we'll be using to find the place ID for the location that we want to get the forecast for so find places API documentation for this apis here let's see what it needs in terms of parameters a text language and an area let's start by adding the text and we'll see from there so basically create a new variable add the text there and then for a location that we want to the location that we want to get the weather forecast for would be like New York City where data Camp's offices are located so let's see what that does it will give us this variable and now we can do pretty much the same thing as we did before um to put in the response object request.get then we need to figure out what the URL is this is this and you copy everything up to the query parameters and then you add the query parameters as a function argument which is PRS is and then metos Source oops there we go the evaluation here for the response code is already there there's already a system exit so we should be able to run this now and see what it does it gives us an error let's see what happens here this needs to be plural prams not singular oops an error occurred something went wrong talking to metor Source same draw again let's print the response text and see what the API tells us what's wrong wrong neither the Key C parameter nor the XA key oh yeah obviously we forgot to add our headers like this tells us that like we need to either add a key to the Cur parameters or we need to have this xapi header but we're not passing our headers to the API call here so what we need to do is add these headers me your Source headers there we go so similar to how you add these parameters group parameters with the prams function argument you can use the headers function argument to send a dictionary of headers along with the request if we run this this seems a lot better no output but that's because we're not printing anything response data all right we've got quite a lot of information back so what we're trying to do is find the um Place ID for New York City so what does the AP documentation tell us um the response can contain multiple places sorted by relevance so that means that we'll likely get more than one response which is sorted by relevance so like it looks then that the best choice is to take the first entry in this API response and get the uh Place ID from that so that's what we'll do let's create a place ID variable um and then extract the data from the response data object uh that would be the first entry of that list and then the place ID value from that and actually we put that here and then let's print the place ID and see what it gives us yep okay this is the place ID for New York City which obviously is New York City um but this is what we'll need to pass along in the other API together what forecast so this is on so now we'll go uh go to the next part of the uh metos Source integration which is getting the actual forecast using that place ID for that we will be using the point weet API Point wer API documentation is pretty elaborate too um but if you read the documentation you'll learn that you'll need to pass along a place ID and a couple of other interesting parameters too like for example you can ask for the current weather situation like this gives you the current weather what's the weather like now at this point in time in New York City or you could ask for a daily forecast which will give you the forecast for the whole day for the next couple of days it seems like what we need basically like we need the forecast for today given that we will run this every morning what we want this to know we want to know what the weather is going to be like this morning and the rest of the day so we'll need a couple of quer uh parameters Place ID is one sections is the second so let's build that dictionary with quer parameters Place ID is the first that's the variable that we've basically extracted here in the previous call and then we've got the I believe sections and that should be let's see you can specify more sections by separating values with a comma so you'll need to pass on a string but we're only interested in the daily forecast so no point in separating it let's use the daily forecast as a section and then what's interesting too is the units uh by default it will automatically select the forecast uh the units of the location that is being forecasted which in this case would be U the Imperial but I prefer the metric one so let's use this also to request the forecast in a metric format and that should be it for the Cur parameters language we can use the default time zone we can also default latitude longitude is not needed because we're Place using a place ID so that should be enough there we we go cendal response equals requests.get and now we need to figure out what the URL part is this is the point API don't forget in to head in the headers again Med Source headers because for this we also need to authenticate and then the prems is uh metaurus Point your out prams we'll evaluate the code whether or not the server responds 200 we'll get response data as Json let's print the response data and this time hopefully no errors yep there we go so what we've done now is we've sent the um location or the place ID and requested a daily forecast in the metric system so if we look at the respon BS oh I accidentally clicked the back button that's not the intended y here we are so basically what we did now is we passed this place ID requested to forecast in the metric system and looking at the response again there is quite a lot of uh data being uh returned we can look at the actual API documentation which tells us for example like okay this is how the data looks what it's being returned um we requested only the daily forecast section so current is not in our reest hourly is also not there daily will be so there's a daily element in the response which has data which is a list and an object or a dictionary for every item in the list so what seems like most interesting is this um summary of the of today's forecast basically so what we need to do is we need to go and fetch the first item of this data property and then fetch the summary and then store that in a variable so let's do that so that would be the weather forecast equals response data and then let's see that would be daily daily forecast yes the data property the first entry in that list which is today and then the summary of that list and let's see if what happens if we print this weather forecast yeah there we go so this is current currently the textural representation of the weather forecast for New York City for today partly sunny changing to cloudly B evening temperatures between 9 and 14 degrees Celsius so now we've got the two pieces that we need um we've got the news headlines we've got the weather forecast what we're going to do now is we're going to go on to the exciting part where we will involve AI uh will you be using the open AI API to generate a textual representation of this uh daily update that we want we're going to feed the news headlights into it and we're going to also feed the uh forecast into it and it will generate a very nice um liely phrased um easy to read a daily update for us that we can read in the morning so what we'll need this we'll need to be able to we need to first of all send a prompt to the API the goal of this uh codal along is not generating prompts so I went theong and already created these prompts um you'll need two in this case you'll need a system prompt and a user prompt basically or a system message and a user message with together will be the prompt system message um is basically telling the um open AI API how you want it to respond like what do you want it to do and then the user message is actually like providing it the instructions with the data that will be used to generate the message so what we can do here is we're just going to copy paste this prompt into the system prompt here system message so this is what we'll be sending as a system message like you're an AI assistant T you generating a morning update text that's engaging and enjoyable for The Listener for the user to listen to while having their morning coffee and so on and so on we're also identifying the structure here for example like you want want you to start with a greeting and then I want you to give me the weather uh summary and then I want you to list the headlines for me and then I want you to close up with a nice remark and maybe a joke at the end um you are asking it to be creative and like stick to a specific tone um is something that you can do too so that's the system message and then the actual user message is where we will be injecting the data that we got from the two apis so we copy paste that into the user message please generate the morning update text in a funny and light tone here's the weather forecast and then what we'll do is we can interpolate the weather forecast let's look at what the variable is that we stored it in where forecast so this will be where the we forecast this and this works because it's an F string um which is a basic python concept you can interpolate variables in here um and this would be the headline articles yes so basically what this means is the message that we'll be sending to the open AI API is please just generate a morning update text in a funny and a light tone I'm asking for a funny and a light tone here here's the weather forecast I we'll dump that weather forecast that we got from the API I'm going to en close that in triple back ticks usually something useful for open AI to distinguish between data and actual instructions and then we're going to add the news headlines uh in adjacent format which is currently everything in the headline articles variable and then generate the text as specified in the system prompt following the structures of the greeting weather summary 10 lines and a closing remark that's everything we need in terms of preparation for the prompts now we'll need to start sending them to open AI let's inspect the documentation on the platform. open AI portal you'll find the text generation capability which um I can actually probably better use the API reference API reference is more the technical documentation so what we're going to ask is using the uh chat completion similar to how we would work with chat GTP right we send it a prompt we receive response now we'll do that through the API um the documentation here is a lot clearer than the previous ones for example like here we see like okay we need to send a post message this time not a get message anymore we need to send the post message to the completion Z point and then there is a couple of variables that we need the messages is a required variable model is required and everything else seems like optional even deprecated for some so let's start with that we'll need to send a API request to this URL and it's a post request this time oh actually we need to first set up the headers which is exactly the same as we did before um checking the authentication and then yeah an HTTP header as follows so it's the authorization and then Bearer and your Opa open API key open AI key so let's do that that would mean that we need an authorization header and for this we need Bearer the bearer keyword is necessary and then the open AI API key that should be it so there's an open an authorization header and the value is beer keyword and then your actual um open API key let's run that and then we need to start uh by compiling this request data this is a little bit more complex than sending a get request simply because in this case we're going to create something new on the server basically we're going to ask the server to generate something new based on the data that we'll be sending there the data is a little bit complexer so we're not going to be using query parameters or this we're going to be uh using the Json property or the Json parameter uh for the request Library so we're basically going to send it like a load of Json data instead of individual uh string values so for this one we'll need to um create a model and create a messages uh property let's see how we do that um the first one is the model property and for this we can use the let's see if I have it in the notes here somewhere yeah GPT 40 mini um fast enough and cheap enough uh for our purposes here so that instructs the server to use this model to generate a response and then we'll have a messages property um let's quickly go back to the chat endpoint great chat completion like messages is a list of messages comprising the conversation so far depending on the M use different message types are supported and so on so what we'll need is we'll need to pass in the system message and this user message so this is how it would look like in terms of Json we can add that here it would be a uh list and then a entry with the keyword roll for the system message and then the content would be the value that we put the system message in which is system message and a second entry we need here which is the RO and that would be a user message and then content here would be user message so based on the API documentation at open API this is more or less the data that we need to send to the server it will tell the server to or the um chat completions API to use the GPT 40 Mini model and it will send the system prompt um or the system message and the user message along as the prompt um for for the um AI API so let's register this variable run the code and now we're ready to send that API request remember we need to do something slightly different here um in this case the response will be request and this time not get but post we're posting to the uh API figuring out what the endpoint is that would be this um there we go pass in the headers which concludes which includes our authentication details um that would be open AI headers yes and then we need to pass in this completions request data but as we said we're not going to be using Cur parameters or we're not going to use the uh the parents keyword or going to use the Json keyword Json equals then completions request data so this will basically send all of the data that we've compiled here so the full prompt including the system and the user message along with the model that we want to use to the server using a post request and that will use the headers to authenticate same drill here we'll evaluate the status code after the response is sent we'll extract the response value and then we can print the um response Json if something goes wrong we'll see the error message so let's give it a try and see what we get an error and that's because this needs to be plural the library is called requests not request that looks a lot better and there we go so we now send all of this information to the server we've received a response the response that we received is not just the chat completion right it's just again adjacent object structured with additional data like it has a chat ID for example um then it has a chat completions object with a creation time stamp and the model that's being used along with the version of the model and then there's the choices for example with like um all of the actual options that we have in terms of like what response to use so in this case it seems like the content property of the first choice is our actual prompt response so we'll try to get that out of the API call and then move on in this case we will say um maybe the morning update equals um response Json and in this case we'll need to look at the choices list that's here then pick the first choice and look at the message and the content of that message print think you want to see in the middle of choices oh yeah thank you I wish I had you all the time for like my second pair of eyes on my code um all right so the choices we've basically got this morning update from the response and there's a lot of additional validation you can do here for example you could look at how many choices did I get did I get only one did I get less than one in that case like there's obviously no reason in doing this so you could for example first evaluate the length of this list for example uh but for the purpose of this code along and time I'm going to continue doing um just this quick uh quick workaround here printing the morning update we'll run this code again it will not be the same output given that we're using AI to generate it but it will use the same data so the same way I forecast the same news updates and this is where we have our actual response message so our update morning update message here is this this is what open AI came up with in terms of a morning update um it includes our today's weather it's classic partly sunny with a side of clouds it's uh the weather to wear pants uh and the temperature is going to range from 9 to 14 degrees and so on I asked it to be funny so it's a little bit more aose but I like a funny approach to my day not too serious and then it's J Jump into the headlines it's going to actually like voice over all of the headlines that we've actually send it uh and then it's the end of the uh morning update and it tells us to have a fantastic day and see you tomorrow so this is basically where we uh could stop but open AI has quite a lot of other useful and cool apis um and maybe to take it to the next level like what we could do is use the text to speech API to not only like have a readable one that you can like read on your phone or your tablet or your laptop in the morning but maybe I want to listen to an update and open AI has this text to speech um API that you can use where you feed in text and it generates um output in a spoken in a spoken Manner and you can save that as an N tree and then listen in the morning so if we would be using that we go to the um audio apis here and you've got create speech and this API in real in in reality is quite a lot simpler than the uh chat completions API um you'll see that the audio speech API this is the path for that API and it only has three required parameters the model that we want to use um which in this case probably tts1 would be enough for us the input which is the text that it will basically generate audio for um and then a voice um The Voice the different styles of voice is female male like high tone low tone basically similar to how you select the voice in your um uh in your chat GPT applications um I believe we could use the Fable one which I prefer personally but you freid to choose whatever you want so looking at that um we now need to craft the same di just to interr so uh before you go through that uh we are coming up to time so just for everyone in the audience we're probably going to run over by 5 minutes or so um if you do have to jump then please do check out the recording and I want to get over get through like one or two questions so if you got any questions for Chris please do drop them in chat now and uh we'll we'll we'll get to whoever answers first we'll ask something first all right sorry Chris continue no worries um okay so again we're going to create a dictionary with all of the data again this needs to be a post uh request as shown by the keyword here uh with a model input and a voice property so let's do that model will be tts1 input will be the response from our check completions so the morning update and then the last one was voice which we call Fable so now we've got this TTS request data property um and we can start by I don't know why this case has an error but um sending the requests so response equals requests and then we need the full domain and PA to the speech API don't forget to pass in the headers uh that would be open AI headers these include our authentication details and then again Json with the with the dictionary we created here TTS request data we'll be evaluating the status code whether or not successful if not will um raise an error I'm not going to run this yet because this is in essence a pretty special response like it will not resp responds with text that we can read or print um in in in the output of a cell uh but it responds with a stream of data that you can save as a MP3 file so you can't really just treat this as a simple um response that you can print and inspect what's wrong basically um you'll need to look at and save the actual data but luckily like a python like storing a stream into a file is pretty simple like you can just use the uh open function um from python so opening a file uh reference by using open let's call the file uh morning update. MP3 something like this important you also need to um pass in a mode for the file so we're going to be writing to the file and we're going to be writing binary data so no text data but binary data to file so WB needs to be passed as a second parameter so now we're basically like have a file ready to and opened where we can start uh writing our data too so what we need to do now is file write and then we need to uh take the response and the content of the response like previously you've seen me use text for example this response does not contain text so this will basically either error out or give you a blank string because there's no text to respond or Worse give you the text representation of an MP3 file which is totally illegible to any human but what we will need is the content of the file of the response basically so we're going to be pushing the whole response content into this file and then what we'll do is we'll close the file and that should be it so basically sending the request with Json instructions which has the uh update message if the response is uh 200 status code everything is fine we open the file write the contents to the file close the file and if we run this and it is successful which is not the case let's see um print response text ah yeah we made an error we only allowed to make post requests so like let's get rid of this and obviously I did the get here that should be the post and the fact that this now takes longer is an indication that it is working uh generating this generally takes a couple of seconds um but once this is done you'll see that a file will appear if you open uh the file browser to your right here oh while we're while we're waiting for that to run do you want to take an audience question yeah sure absolutely since we're sort of well past time now let's let's get through this so uh adre was asking um how do you use an older model I think you sort of covered this with like specifying the um the gbt 4 mini version but you want to talk about like how you decide which model to use like when do you want a smaller model when do you want the the bigger fancier models um it titly depends on like what you're doing basically this is a very simple use case where a lot of like creativity is allowed like we're not being very deterministic in what we want we wanted to be creative in the response um and for the purpose of this like I simply chose GPT 4 or mini in this case because it's the cheapest model too um that is basically the only motivation for me right now but it depends highly on the task that you're doing like if it's about generating a trivial or a frivolous morning message gp2 40 is is is good enough um the 01 models for example are a lot better at like complex reasoning so if you wanted to do a lot more complex reasoning in the response then you'll probably better off using one of the other models okay so in general it's like start with the cheapest model and if the performance isn't good enough then consider using something more expensive yeah yeah yeah I also probably would not start by coding everything right like you can use most of the models in CAD GPT um like give it a try do some prompt engineering try and figure out like what works what doesn't what models have the best output and performance um you can even go a lot further with like on the code level but as as exploration exercise it's probably best to use check GPT first um to decide what model and prompt works best and then move on and put everything in code okay I like that uh use the interactive interface first and switch to code all right uh is your um is your audio thing run uh I'm just conscious of time it has yes it's done um and there's the morning update message here so if you open the file browser here and move to the files tab you can now see the morning update and what you can do now is download the file and run it I don't know if I'm sharing audio so I don't think it makes sense uh playing it um but now you should be able to download the MP3 file and listen to it and it will like voice over the complete uh morning update um that we've generated by using the completions API here um that's about it um this is where I wanted to like wrap it up um there's obviously a lot more stuff that you can do here um you could for example tweak the API prompt a little bit to be more motivational and with a joke for example um but it becomes really more interesting when we're talking about like using other apis to also feed into this you could for example use the Google Calendar API to give them an overview of your meetings of the day or or the Gmail API to give you an overview of how many messages you have or maybe a summary um there's a million of apis available and one really useful resource I want to leave you guys with is uh these this public API um GitHub repository managed by API layer it contains a huge list of free apis that is like manually curated by the people from API layer um and it's a list of apis that you can free freely use like with a lot without a lot of cost experiment with um like it contains a a list of of of a lot of useful things um so yeah that's that's where I want to leave it at um I don't know if there's any further questions or all right super uh yeah we don't seem to have any more questions uh so uh thank you so much Chris that was very very cool stuff and I do love the flow of just like you pull in data from wherever you want bring it all together feed it to uh feed you to an llm and then you get something cool out of it uh but in general yeah it just seemed like uh I mean I said this at the start but working with API is one of the most important skills for anyone working in data anyone working in software uh so yeah I hope everyone learns a lot now if you're interested in this uh so tomorrow and Thursday we've got a two-part session on building AI agents which is basically like the stuff we've been doing today but taking it a step further so uh please do come back for that and then as you as Reese mentioned we've got a YouTube only session on Friday uh so yeah lots to learn about please do come back later this week and uh use the QR code to register for future events all right see you again soon by