Personalize conversational AI phone calls with Twilio

Key Takeaways

we just launched a new feature to allow you to personalize your conversational AI agents to whoever is calling them now the way this works is via setting up a web hook that returns some Dynamic variables like the customer's name their account status um as well as a prompt that you can inject and the first message to the customer as they call so for example you know if I'm calling I want the conversational AI agent to say hi Thor um how are you thanks for calling you know how are things so now to recap first of all how can we connect our you know conversational AI agent to phone calling so the way we can do that is via twilio and so if we have a twilio account we can import an existing twio phone number into our conversational AI agent so when we go over to our conversational AI agent dashboard you can see there is uh a setting for phone numbers and so here I put in my twilio phone number already so this is the phone number that folks can call my conversational AI agent on and you can see here you can specify the inbound calls um to Handle by you know which agent so anyone who calls this phone number should go to this twillo test agent so let's have a quick look at my agent configuration here so it's configured in English um you can see there is some default first message um that just says hi I'm Thor um and it's using actually my voice clone so you know if you're lonely you can just call uh and you can talk to my voice clone clone if you want um but let's have a look at you know what this looks like if we call this agent hi I'm Thor a developer experience engineer at 11lbs what do you want to learn about hey Thor what is my name I do not have access to that information is there anything I can help you with uh Regarding 11 Labs so yeah you can see that you know that is the experience want to improve so we want to inject that personalization data into the agent so you know depending on who is calling so the way we can do that is by just setting up um a little web hook so for example if we're using um JavaScript we can use fastify to just set up a web hook Handler and I just have a route here so um twilio basically when the call is being initiated ated and you can you can see that here so while the call is being initi initiated in the background the conversational AI agent so 11 laps is making um the post request to your uh API endpoint to your web hook uh to get the information and then you know once the the call is being picked up all the information is read readily available so you can see here you know um you will get uh the here the caller ID from the request body so that's being injected and you can then you know look up in your database or your integration you know your stripe account what have you the information about the customer and then you can inject that into um the conversational AI agent you can see here now that I'm overwriting this uh to hi John how can I help you today uh and you know I mean this prompt is not very helpful but you know they can ask about like their bank account balance okay so now the way we set this up is we need to configure our web hook details so in our um settings here we can add a web hook for the conversation initiation client data web hook uh and we can also add some secrets you know if we need them in our web hook um so here I'm using enro to expose my um local URL to so you can see my server is up and running um to my agent so here and then my route I called it get personal so we can just say Get hyen Personal uh so we can add some headers you know if we want to verify kind of a secret um you know ideally once you move to production uh you want to do that um but for now we're just going to add this Po in here uh and then the next step we need to do is we need to um yeah so we can specify our headers uh we need to enable the fetching of the conversation conversation initiation data um so you will have one web hook for all your agents so when you go to your agent um you know the webook will have the agent ID inside so if you have various different Agents set up um you can have hand that there and so then here in the security you can say Okay enable fetching of the conversation data you need to save that here uh and then you can say Okay first message and uh system prompt what do you want to allow to overwrite um and so you can see we're passing in here the prompt and the first message and so now when we are initiating our call again [Music] hi John how can I help you today hey oh how do you know my first name I have access to some basic information to personalize our interactions is there anything I can help you with today John yeah what is my my bank account balance do you know yes John your bank account balance is $100 is there anything else I can help you with regarding your account yeah so this is so fun talking to myself cool so now you can see that as the call was being initiated from twilio um the weapo call was being made in the background we got the the body here the caller ID the agent um the number that was called as well as the uh calls ID so and then based on the caller ID we can you know fetch the information and return it back so there you go this is how you can make personalization happen with your conversational AI agents over the phone uh let me know what else you want to learn and I will see you next time