Python Tutorial For Call Center Analysis With AI Using Speech-To-Text

Key Takeaways

hello everyone in this video we're going to be building a very simple python application which makes use of machine learning to analyze customer calls to do this we're going to be making use of assembly ai's powerful speech to text API as well as their llm framework lemur to start off let's take a look at what our end result will look like here is a customer call about a missing package according to the FedEx tracking your parcel was delivered on Monday and left at your front porch That's Impossible my apartment doesn't even have a front porch also I was home all day on Monday and no one attempted to deliver a parcel and side by side you can see our application making use of lemur to retrieve action items from the call and this is an excellent real world use case which is used by a ton of different companies to start off we are going to be coding the logic of this application in Google collab so it's much more easier to run and understand and before we do that to make use of assembly ai's API be sure to go ahead and create a free API key which we'll be using in the application and the link to the API will be in the description box below let's take a look at our application and analyze the core logic behind it the first thing we're doing is selecting an audio for analysis what this does is it allows us to upload an audio file to assembly AI speech to text API which then transcripts that audio file for further analysis and that is exactly what our first function here sent to API is going to do the second thing that we're looking at is sentiment analysis and that is what our second function over here analyze sentiment is going to do the third thing that we're going to be doing is generating action items by making use of the Lemur framework and that is what the third method over here is going to be handling so get action items is going to be generating action items from assembly ai's lemur framework to start off the first thing we're going to do is install assembly ai's python Library into Google collab and we can do that easily by running pip install assembly AI once we've done that we're going to get started on our first function send to API the first thing we're doing in the center API function is defining our API key and this right here is where you should be writing your own API key which you can get from your assembly AI account foreign [Music] so this first function sent to API is complete and what we have essentially written is we have created a transcriber object which will help to transcribe our audio file and once we've done that we've created a transcript variable where we'll be storing our actual transcript that we get from assembly eyes API once we have gotten that transcript what we're going to do is we're going to pass it into our two other functions which is analyzing sentiment and get action items and these are going to be giving us sentiment values and retrieving action items by making use of the Lemur framework once we've done that we're going to return a Json object which includes both the sentiment that we get and also the action items and essentially this Json object is going to make it really easy for us to pass this data into our application UI later on foreign [Music] so this is the analyze sentiment function and this is how it works so assembly AI speech to text API returns a sentiment value for every single sentence which was spoken in the audio file that we sent it and what this function does is it counts all the positive instances and negative instances so whichever is higher that's a sentiment that we're assigning to the entire call and this is the basic logic of this function and we can hit Run next up finally we have the get action items function please [Music] so this is the final function which retrieves the action items from lemur and how it works is we pass our transcript to lemur by calling transcript dot lemur dot action items to specifically retrieve action items because that's what we're looking for besides generating action items lemur can also do summarization as well as question and answering so that can be something interesting to try out as well so when we're passing a query to lemur it's best to provide some sort of context on what your transcript is about so that is why I have included this context that this is a transcript of a call between a customer and a call center agent and also I've given it an answer format in which I expect an answer to be returned so a topic header and followed by relevant action items once we've done that we're going to create a return statement and hit Run next we're going to run the cell by actually calling the send to API function and this time we are going to be passing in our audio file as input for you to analyze and generate all the things that we need like sentiment analysis as well as the action items once that cell runs we're going to print out the response from the API so as you can see the sentiment for this call was mostly negative and lemur has also helped us to generate a lot of important action items which can be used as follow-up so now that we have completed the three main functions that we require for the logic of our application so we simply need to copy these three functions for it to run to download this project file all you have to do is click on the GitHub Link in the description box and you should be able to download the entire project file and what we're going to be doing first is once you open that project file in vs code click on the terminal and what we're first going to be doing is creating a virtual environment and downloading all the required python libraries for this UI application first if we're going to be creating a virtual environment and I'm going to be writing down this command to create a virtual environment once we have done that type in this command to activate the virtual environment that we have just created and once that is activated you should see you should see the name of your virtual environment right here at the beginning of your command line and then what we're going to do is we're going to download all the required python libraries which is actually in the requirements.txt file so just do pip install requirements.txt and this will essentially just download all the python libraries that we need to run our application once that is done let's head over to our application file and click on AI bot dot pi and this contains all the code for our application so as of now I have copied the exact same code that we have written in the Google collab into our project file the only differences that I've done is I've added self to each function to the beginning of each function as a parameter and also self over here as well and once you have done that it is ready to go you can hit run on the application to actually test this out and here's our main application you can go ahead and click select audio fire analysis and I'm going to be selecting an audio file and click open and here we have the final results so we have the overall sentiment analysis as negative since we have given a negative audio file and also with the help of Lima we have managed to generate 18 different action items for customer service the delivery process for call monitoring and Drop Shipping and this was incredibly easy to code with the help of limer's framework and the API let us know what you thought about this video in the comment section below and if you want to see more use cases for lemur you can check out the documentation page which I'll be linking in the description box as well thank you guys for watching And subscribe for more amazing AI content