Build A Talking AI Agent with Claude 3.5 Sonnet - Python Tutorial

Key Takeaways

hi Mother oh sweetie there you are I was starting to worry have you eaten anything today you know how important it is to keep your strength up don't make me come over there and cook for you I had a bagel have you heard anything of Alice oh a bagel just a bagel that's hardly a proper meal sweetie you need more nutrients to stay healthy but never mind that for now did you say Alice let me check if I've heard anything from her give me a moment dear oh honey I just checked my emails and yes I did hear from Alice she wants to know if you'd like to go to the cinema with her on Wednesday now isn't that nice but more importantly are you sure that Bagel is enough you really should eat something more substantial maybe some vegetables or a nice home-cooked meal mothers in this video you'll learn how to build a AI mother from scratch since we have already buildt a girlfriend hi there I'm good how are you and a bad girlfriend why would I care about meeting you what could possibly be interesting about our conversation we're taking a different approach this time unlike our previous implementation we'll build everything using the most advanced language model on the market Claude 35 that will make the style and responses incredibly realistic oh honey you'll always be my little one now don't you sass your mother I'm just looking out for you and like a real mother she can do more or less everything for you I'll show you how to equip the AI model with various abilities such as checking your email inbox or accessing your calendar by using claud's function calling I'll be honest before starting this video I had no idea how to convince people who aren't as nerdy as I am about how cool the new function calling from Claude really is I watched tons of other YouTube videos about Claude son everyone was talking about the new Claude UI with artifacts but nobody seemed interested Ed in using the API and exploring its possibilities the few videos I did find were hard to follow and I knew there had to be a better way then one day I looked at my video statistics and realized that my best performing videos by far were about Jarvis and AI girlfriends but I couldn't bring myself to make another girlfriend or Jarvis video I was on the verge of looking for a new topic but then it hit me like a ton of bricks the world needs a mother Ai and thanks to 11 labs and Claude Sonet with function calling we can make it happen what is function calling function calling in llms means the model can use external functions or apis to do tasks it can't handle by itself like reading and sending emails data retrieval or accessing the user's calendar the results of these function calls are then interpreted by the language model just like a real person would do now let me tell you what I found in your emails did you forget about dinner plans and what about the movies on Wednesday you're not too old for some family time are you and if you prefer a Never Satisfied father you can easily change the system prompt and the voice hi Dad I've have more than 12,000 subscribers only 12,000 your cousin's dog's Instagram has more followers than that when are you going to start taking this seriously if you haven't worked with python and visual studio code yet please check out the introduction video series on AI forevs how to work with python and apis so let's get started we start by setting up a virtual environment to cleanly separate the dependencies we activate the environment and then create a file named app.py we install anthropic 11 labs and pame all right now we can start by importing anthropic we create cre an anthropic client to facilitate communication with the anthropic API then we create our system prompt you are an AI designed to emulate the behavior of a stereotypical nagging mother your primary role is to remind the user of their tasks chores and self-care routines in a humorous yet slightly annoying manner we continue by giving our agent a short-term memory for this we create a variable called conversation where we will store the entire conversation since we want an ongoing conversation we create a while loop that continues the following processes until it is eventually stopped we save the users input in the variable user input and add the user's input directly to our conversation variable now we add the actual call to the anthropic API we use Claude 35 Sonet and provide our prepared mother AI prompt as the system prompt and as the user prompt we include the entire conversation then we want to extract the response from anthropics Return result and for this we access the message content text and we output this directly to the console we start the script with python app. Pui and say hi Mom but we get an error message because we are not yet authenticated against the anthropic API for this we go directly to the console anthropic tocom page and create a new API key we give it a name create it and then we can copy it and ex ort it as an environment variable in the console so that it is available in the code we start the script again write hi Mom and get a great response like from a typical nagging mother all right now we want to deal with outputting the text of speech and for this we want to use 11 Labs let's first add the AI mother's response to our conversation so she knows what she has already told us now we can focus on creating a voice for her we create a new file called U.P and add the prepared code that you probably already know from other videos essentially there are two methods the play audio method which can play MP3 and wav files so we can actually hear them and the same method which does text to speech that means it can convert the text from the language model into a sound file using 11 Labs we can now import the same method we then use it to read alloud the output from anthropic before we can run the script however we need to set an 11lbs API key as an environment variable for this we go to the API key section on 11lbs and can copy the existing or newly generated API key then we set it as an environment variable and now everything should work we say hi Mom again well hello there sweetie have you remembered to brush your teeth this morning you know how important dental hygiene is oh that's good to hear honey but did you make your bed yet a tidy room leads to a tidy mind you know so now we want to deal with how to give the agent access to tools like emails the idea is that the language model gives us a stop reason when it wants to use a tool for this we need to check if the stop reason tool use was set if that's the case we want to call a new method that can retrieve emails for this we create a new file named mail utils and add a fake method that simulates fetching emails you can easily replace the get emails logic with the specific code for your email provider if you have a Gmail account account you can find a solution in another video of mine Linked In the notes we have two emails one from Bob at aif for dev.com asking if dinner tonight would work and one from Alice asking if we want to go to the movies on Wednesday we also need the so-called tool definition we give this to the language model to explain that there is a method called get emails and through the description we say what exactly can be done with it and through the properties which attributes can be set in this case it is possible to give the name of an email sender so that the emails can be filtered we then import the get emails and the get emails tool definition in our app.py and can now directly use the get emails if the stop reason tool use is the case we then of course want the emails to be understood and interpreted by the language model again therefore we need to send the conversation back to the language model model to avoid redundancy we want to encapsulate the call to the language model in its own method we simply copy the call and create a new method called call Claude which receives the conversation as a parameter we paste the content and now have a method that performs the call for us we then return the result and for the language model to know that this method even exists we need to send the tool definition along with the message so we can now use our call Cloud method both at the top and bottom of the loop as a last step we let the response from the language model be read aloud that should be it we start our script oh and we get an error message that we probably forgot a comma after the tool definition we quickly add that let's try it again oh sweetie there you are I was just about to call you have you eaten your vegetables today you know how important they are for your health oh honey you'll always be my little one now don't you sass your mother I'm just looking out for you have you checked your emails lately I bet there's something important you're forgetting about let me take a look for you oh sweetie pie age is just a number now let me tell you what I found in your emails did you forget about dinner plans and what about the movies on Wednesday you're not too too old for some family time are you