Build Cool Stuff with Anthropic Claude 3.5 Python API

Key Takeaways

so in today's session we will be focusing on learning clae 3.5 Sonet this is very new model it got released like last month 3 Four 3 weeks old this is much more powerful than almost compared to all the models this is the most powerful model this is cheaper faster and more accurate uh on X I'm seeing lot of people Started Loving like Cloud models they are getting amazing results compared to even chat GPT things like that also these kind of models are available on both Amazon bedrock and Google Cloud Vex a for example let's say your workplace uses AWS you can access these models in a much more secure way also if you whatlace uses Google Cloud we can also access the models on Google Gardens sorry metx a has model Garden you can access it there Cloud 3 Hu and CLA 3 Opus and CLA 3 Sonet they are like their previous model recently they launched this model CLA 3.5 Sonet which is like their previous midlevel model but if you see the performance cost and performance this is like very good spot compared to Cloud 3 Sonet and Cloud 3 Opus soon they are planning to release Cloud 3.5 hu and CLA 3.5 Opus so CLA 3.5 Opus somewhere should be somewh here like even more intelligent and effective than previous models I've been using this model since from the day one of this model launch I really like the model it's amazing it does a lot of work it's very helpful I'm reading a few messages Frank asking can this be recorded and share later and mother train and Harley here okay he says he couldn't hear much so yeah this video will be available on the YouTube channel you can watch it later too I'll try to pre-process and increase volume if needed but I hope you can hear me better right now okay if you want me to increase volume let me know I can do that cool so if you see the models metrix it's CLA 3.5 son is compared with CLA 3 Opus gp4 gem 1.5 Pro and llama 400 billion parameter model if you see the results plot 3.5 son it is amazing in almost all the metrics other than one which is not too bad this is really nice and the one the thing I really like about this model is this is much more faster than gbd4 and Opus and cheaper and more accurate so that's the main reason I started liking this model also recently This Cloud company like they are doing lot of things every day if you see see their API and developer dogs they introduce a lot of cool things let me quickly log in so we will see all their new features and we will try to understand how to use these models using python uh just uh I launched your course on uh udmi on mastering the anthropic clot 3.5 by python API I just launched this course I haven't started publishing it anywhere it got released just today this morning I can share the link on our community and also if you are interested in learning more about that just message me I can give you like free coupon it should be 100% free you can learn it so I can share that on our Channel I'll message you the free coupon just message me on YouTube or just here should be fine okay so in today's video what we are going to do is we will try to understand messages API a little bit more clear also we will do a quick side project this one like using fridge let's say you have a picture this picture and you want to know what are the things you can make with this picture let's say you take a picture of your fridge and then you want to create a recipe sometimes the model will also recommend one or two products that you can buy so that you can aim for lot of recipes so this is helpful in generating recipe uh without wasting a lot of your food in the fridge you just take a picture and you upload and you get all the results cool I'll talk about the messages APA and then we can move to this one so messages API is like a foundation for interacting with their anthropic models to install anthropic you can use bip3 install maybe let me Zoom a little bit so in today's session we will focus more on model Cloud 3.5 Sonet and especially this is their first party API model name one piece first party and they also have other parties like AWS and gcp so we will focus more on first party which is anthropic and we will focus on this model Cloud 3.5 Sonet 2024 0620 so we will use this so to install this model you can use pip3 install anthropic I'm great let me quickly run this I think the recent version should be 0.31 okay I think my version is already upgraded and you can also install pip 3 install python. environment the reason is you have to pass the API key somehow to the model so instead of directly passing the AP key like a string what you can do is you can use environment file and in the do environment file you can add your API API key here like to find an API key what you can do is you can go to dashboard console. anthropic doccom D dashboard C C get API keys and here you can create API key and you can type something and then you will get some key cool and you can copy the key and paste it here so that's uh faster so after pasting the key what you can do is you can run this one input OS and from environment inut load. environment file so when you run this function this will load all the things in environment file into your terminal so that into this python environment so that you can access the AP key like this.g enironment anthropic _ API key so that AP key will be stored here and then you can pass the API key to anthropic do anthropic AP key and you can pass the AP AP key here so once you pass the AP key now you can interact with the model to interact with the model you can use client. messages. create so this is the main function client. messages. create and it needs three things minimum one is model what model name we are going to use and Max tokens and temperature so these three things are important if you don't have these three things any if you miss even one of these things the model won't run you will get the error message sorry model max tokens and messages these three are important messages Max tokens model three things optionally you can set a few other parameters too one parameter is temperature other parameter is system we will learn each parameter in detail in today's session so I'm not going to run all the code but I can share the code you can also run after the session I think you can focus more on understanding the building components how to get some answer how to change the results things like that cool so if you run this one here I'm asking top 10 places to visit in Sydney and if you see my messages it has two things role and content role could be user or assistant when model rep something it could be assistant when you ask something that's user so we have two things user and assistant always it should be alternative first it should be user and second it should be assistant then the third one should be user things like that also Max tokens most of the models like 2 days ago they support 4096 but very recently like yesterday I think yesterday they increased the max tokens to 8192 tokens 8192 so 8192 is the max token let's not focus on this right now just use th000 token even th000 is good for these kind of use cases cool so the most important thing here is you have to use client. messages. create and it needs three things minimum one is model the second one is Max tokens the third one is messages and in the messages it has two things user sorry uh role and content role could be user or assistant you cannot have two user message consequently like next you cannot have that it should be alternative between user and assistant then user then assistant like that cool maybe if you want I can run this maybe I can create a AP key and then delete the key after the session copy close and paste it here okay now I'm going to run this so now if I check my AP key I can see the real key AP key and oh it's not there restart GL I checked okay since I mod modified that do environment file I'm restarting my system like restarting the environment after restarting I can run this so and AP key print first 10 characters and you can see there is a AP key there now I am creating client then once client is created I can use client. messages. create and here I can pass the model max tokens temperature and run this okay now I can print so to print I'm using I python. display input display and HTML markdown so this is just a basic display so as you can see Cloud family has four models these three are older and this is the most advanced and recent one most intelligent model and if you can see all the models are multilingual you can pass any different languages and you can get response in different languages and all the models are Vision capabilities you can add images and text together to get some response and this is the model name and you can see messages API is the format and it's very fast which is good then context window is 200 tokens which means you can input 200 Tok sorry 200,000 tokens to the model which means it could be like at least 500 pages book so he he can pass 500 pages of book to the model and get some response maybe let me quickly ask how many pages book is roughly 200,000 token let's see approximately yeah approximately 550 to 600 pages long so which is amazing like you can input 600 Pages book to the model and you will get some response so the response is as you can see this is like older one they just changed the max output to 8192 tokens they doubled the max output so 8192 token is approximately 22 Pages book so you can pause 600 Pages book as an input and you will get at least 22 pages of Word document like an output which is really great which is really good for uh programming like coding you can generate lots of code with that and you can see the cost so this is like 1 million token so to process 1 million token this is $3 and to Output 1 million token $15 and you can compare the price then their knowledge cut off is April 2024 which is the recent model and all their old old models knowledge cut off is August 2023 so knowledge cut off means that some date there is a cut off until that date they have the training data so to train this model they used all the data that's available uh on the web up to April 2224 cool so this is their old model clot 3 Haiku to use different model just change the model name and rest you can keep it simple and run this so Hau is super fast so you can see we are getting the response in 3.6 seconds compared to previously I think we got and it's not displaying let's see this is their large model Opus so that's why taking longer 13.9 seconds but if you see our previous one CLA 3.5 Sonet which is relatively fast like just 4 seconds 4.3 seconds cool so now let's focus on messages APA parameters maybe let me quickly go through each in detail and then I will explain these things so as you can see we need three things as parameters they are model you have to specify what model you have to pass and and messages is a list and this array of messages object present in the conversation history the conversation history should be user assistant user assistant like that the first message is always user then when you you when you add assistant and then you start some message that's called pre-filling we will see pre-filling in the Json lesson this on messages AP Json as you can see Ro user some content then assistant then some content then user then some content so user assistant user so if you use two user at the same time you will get error message and this is called partially filled response which means user is us we are asking some question what's the best way to see the Sydney Oprah house and then if you ask if you want the model to respond by starting this to get the most out of your visit to Sydney opa I recommend then from here you will get the models respond so we are free prefilling something this is called pre-filling and we can also pass images to the model we will see how to do that let me quickly see if there is any questions cool okay so now let's try to learn Max tokens meanwhile wa second I'll try to open one more think okay so I think this looks good so let's start with Max tokens so currently Max to tokens could be up to 8192 so here we are using like 1,000 tokens for 1,000 tokens this is like a maximum token the model will cut off after generating th000 tokens but sometimes it won't this is like a Max cut off like a it won't generate more than thousand but it's not guaranteed to generate up 2,000 sometimes it it could answer in less tokens in this question we are asking just only top 10 places to visit in Sydney it can finish that in few tokens like 200 tokens so let's run this and to see the response you can use message. usage so let's print this one too I'll show you how the message object looks like okay maybe let me Zoom a little bit so this is how this looks like so there is a library called Rich I'm going to use Rich to print the output from Rich import print yeah so this is a library you can install pip sorry pip install Rich then you can do this so this will show you things bit bit more clearly so it will be helpful especially working with message output puts so as you can see the results there are three things message text block and usage three classes and message class it has like ID and content has text then the message it has few other things too model what model we used and what is the role the role is assistant because the model is responsing that's why it's assistant and why it stopped it stopped because it reached the end turn which means it completes the answer and stop sequence none so we will see what stop sequence mean in the next one then this is a message type and you can see usage in the usage object you can see two things input tokens output tokens and message. usage uh here you can see input tokens and and output tokens yeah so now what I'm doing is in the previous one we said max tokens and thousand now I'm setting Max tokens as 100 and see what happens so if I set max tokens as 100 the model will stop and when it's about to reach 100 so it will stop like 99 so you can see when it reaches like 99 tokens it stopped it didn't it stopped abruptly it didn't continue and now if I print message you can see why it stopped it stopped because it reached Max tokens stop reason is Max tokens previously why it stopped this it reached the end turn which means it completes the answer now it didn't complete the answer it stopped abruptly because we set max tokens as 100 it reached 99 tokens and stopped and you can see the stop Reon cool so now we are going to see what is stop sequences so use stop sequences to Define custom text that stops the model so you can this is like a list in that list you can add any number of string as soon as the model sees this token it will stop so normally as the model stops at end turn but we can force the model to stop by setting the max tokens or setting some stop sequences things like that so here let's run this so here it prints up to 04 then after completing the point 4 it sees five and the five is in our stop sequences as soon as it it sees the fifth point it stops and you can check the stop reason print message let's run this and you can see why it stopped is stop sequence this is the reason for stopping and what is the stop sequence the stop sequence is five so this is a way to control the prompt like response so usually the stop reason is and turn so when it finishes the answer the stop reason could be like end turn and the stop sequence is none so that's why it's not displaying anything but when you forcefully stop it you can see the stop sequence is some something that we mentioned in the list and stop reason is stop sequence I hope this is clear so now let's move to temperature before that let me quickly check looks good cool so temperature is a way to control the model's output so amount of Randomness injected into the response so if you need always a deterministic answer you can set temperature as zero if you set temperature as high number like close to one the output will be little bit more creative like chooses a little bit more words choices so here what I'm doing is I'm doing some analysis but I'm not going to run this but I will show you the results because it will take a few minutes two to five minutes to run this because I'm running for these temperature temperature 0o temperature 0.7 and temperature 1 I'm running the same question 50 times and I'm I'm keeping the response in the outputs uh dictionary you can see it here so if you run this and then display everything before that in my question is top one famous cuin inia just list the name example ice cream things like that so after running this 50 times for temperature zero even after running this like 50 times I'm getting briani because when you set temperature at zero the output is always deterministic you will choose the best top probability value when you choose temperature 0.7 it will be a little bit more creative but when you choose temperature close to one it will be more creative and chooses a little bit more different results here you can see briani full stop and there is no full stop now there chicken curry things like that so that's temperature so my suggestion is if you need a deterministic answer choose temperature as zero if you want your output little bit more creative choose high temperature like 0.9 or one cool so now let's move to top P so this top P parameter is also called nucleus sampling so nucleus sampling involves selecting tokens based on their cumulative probability so what does that mean so let's say actually this large language model all generative a based models are like predicting next tokens so each token is like a word let's assume these are different tokens available token a and it has some probability 0.5 token B's probability is 0.3 token C probability is 0.1 and D is 0.05 and e e is also 0.05 and let's assume we are selecting top p as 0.8 so what happens is initially for the first word the probability is 0.5 then we have to do cumulative sum so first token and second token the probability is 0.8 since we set top p as 0.8 the model will consider only the first two tokens and it will randomly choose one of those tokens it will skip the remaining tokens like C D and E I hope this is clear so that's called top p is also called nucleus sampling so here a name one famous IND dis in the same question and we are running this experiment 50 times on each different value 0.1 0.5 and 1 cumulative probability and here you can see when top B 0.1 it chooses the highest probability these things when it's like top BS one it chooses the second best one so actually as you can see temperature is bit more helpful when controlling the outputs they recommend like anthropic and even open AI they recommend these things for advanced use cases typically adjusting temperature is sufficient and yeah also they recommend adjust temperature or top B not both so for your use cases just play with it change temperature high temperature and see how the results look like low temperature and see how the results look like things like that so now we are going to see top K so top K means how many tokens you are going to consider let's say you your Lang large language model you are considering like 10,000 words so which could be like approximately 15 to 18,000 tokens but we don't have to consider all the tokens we can consider only the top 10 highest probability tokens or top five highest probability tokens so here what we are doing is we are setting the experiment the same similar experiment I'm considering the first token I'm considering the top 20 token and I'm considering top 100 token and then I'm running the experiment and you can see when top K is 20 I'm getting a little bit more different results when top K is 100 a little bit more different results but when top case one always bran so you can see if you want your output to be very deterministic set K top K as one if you want your to be a little bit more creative choose High high topk so my recommendation is don't use all these things together just play with temperature and see how your results look like then you can change one by one and control the results too I hope this is been clear and helpful now what I'm going to do is I'm going to show you this project since what are the things we have learned so far that's that knowledge is enough to do this project but yeah slight changes we are adding image I will tell you how to do that but I have prepared a dedicated lesson for images as you can see the first lesson is all about messages API then in the second lesson we will cover some of the uh best practices and ways to ways to do streaming which means you don't have to wait for your results as soon as you run the code you will start getting the results then in the third one we will cover all the all things about images we will also do a few many projects in this lesson so we we can do that in two weeks then in the fourth one this is like how to use message let's say you have some unstructured data and you want to convert those unstructured to structured you can use Json mode to do Json mode you have to set assistant and then you can start some pre-filling like the things we learned in the initial lesson so this is how you do some pre-filling you say assistant hey assistant start with this one here is the Json requested and you start with this bracket then the model thinks okay you are requesting Json and you are starting with the Json like this notation then the model will respond continuing from there so you can see it here so this is how you convert unstructured to structured and there are few other ways to do that but this is one quick way and then in the fifth lesson we will learn about tools how to add external information to the model so that the model will decide what external data it has to use to answer your question and it determine everything dynamically you don't have to control you just pass all the tools to the model and model will determine what when to use what then in finally we will do some uh project so I'm going to show you this one fridge so this is very simple this is very similar to what we have covered so far I'm loading all the input statements then I'm loading my key then uh client setting the client so one thing we haven't seen today is how to use im images I'm going to teach you a little bit high level now and then in the future lense we will go in depth so to do images what you can do is this is an image and you can load this image as a base 64 format because these models needs only string so you have base 64 model sorry base 64 representation is a string representation so to do that we are using this Library b 64 and here you can pause the file name and then once you loaded everything you sorry you decode you decode it and it will be like basics of a format which is a string format then here def get diet preference we are asking the users preference here I'm setting as gluten free then this is the prompt I'll tell you how I generate the prompt uh this looks a lot I didn't type this so this is the one new thing they introduced very recently if you go to workbench I think dashboard dashboard this one generate a prompt so here you can just type your very vague idea write me an email something like that draft an email respond to customer complaint email and offer resolution so this is like a vague idea and then you can ask it to generate a prompt now the it uses CLA 3.5 model and then generate a detailed prompt So based on my experience so far I'm getting very good results when I do like a detailed prompt rather than uh a small prompt as you can see it has like XML U tags and this is very proper way of doing things and it thinks everything clearly and it's very detailed so instead of writing like a small prompt if you do in this way you might get even better results so that's how I generate this initial prompt here I am pausing only the dietory preferences and then the prompt isan here I'm asking a few things like uh list list identified ingredients with the estimated quantities and expir dates and provide three creative recipe ideas using near near expir ingredients with brief instructions then offer two to three quick storage tips for maximizing ingredient lifespan then leftover ideas suggest two to three creative ways to use leftovers or food scraps then small shopping I recommend 3 to five ingredients to buy that compliment existing items then some Scopes things like that then as you can this is maybe let me show you from beginning so here you can see messages we have to pause three things model max tokens messages the messages could be user as you can see it's a role user and in the content I'm typing the image content and text content content so there are two things so the first one the image content to do that you have to set the type as image then when you use image the syntax is you have to use source and you you can say base 64 because this is a base 64 format and the original image is jpeg type jpg so jpg and here you can pause your base 64 image string this is a string data then in the second one you can pause your initial message so the initial message is this all your prompt along with your user preferences then yeah then I'm checking the model will run after the model runs you'll get the output response. content 0. text then I'm displaying the results so here you can see it analyzed everything and also since it has like tags we ask it to provide lot of tags we can extract these tags and then create some streamly tab so here here you can see maybe I'll create this as a blog post and then I can share this with you for right now this is very simple idea that can be achievable just by knowing how to do messages API cool maybe I think I'm pretty much covered what I thought of doing today message apaa I hope this is helpful see you all next time in the next time we'll focus more on a project end to end along with some streamly app I hope this is helpful and last time if you're interested in this course just message me I can give you a free token yep I'll share the code in our slack Channel yeah thanks everyone see see you