How to do content moderation on audio files in Python

Key Takeaways

hello and welcome today we are going to learn how to moderate what is being said in audio files in python so this is what our results are going to look like here we have a list of parts of this audio file where something is being said that might need to be moderated out we get the text of what is being said we have the confidence level for this label what kind of uh problem there might be with this part of the audiophile and the severity of what is being uttered so here it says crime and violence but the severity is very low so maybe if it was 0.9 there could be something about crime or violence that could be very disturbing to the listeners so let's build this so to achieve the content moderation in this project we are going to use assembly ai assembly ai has some audio intelligence features and content moderation is one of them and that gives us the results that we've just seen so to do that first we need an assembly ai api token so let's go get that you can either use the link in the description or you can go to assemblyai.com create an account through get started and once you have an account you will be automatically assigned a free api token that you can copy right from here and here in our project we have a main file where we're going to have the actual code and in the configure file i'm going to keep my assembly ai authentication key and once i've done that of course i also want to import it here and to talk to assembly ai we are going to need to use the requests library and it's actually very simple to start a transcription or audio intelligence job with assembly ai all we have to do is to send a post request inside this post request we are going to need three things the first one is the endpoint of course this is the endpoint that we are sending the request to and we are going to use the transcription endpoint of assembly ai and the next thing that we're going to need is the data that we're sending to assembly ai you can either upload your data or your audio file to assembly ai or if you like you can upload it somewhere else and just pass the url to assembly ai which will be a faster way of doing it so that's how we're going to do it and it's going to be in json format so i'll also just call the variable json to keep things simple so the first thing that i need to pass is the order url and also in here i want to include what i want assembly ai to do on top of the transcription and here we want to do content moderation so we want to turn the content safety feature of assembly ai on because normally if you don't specifically tell them to be true it by default is false all of the content miteration sentiment analysis and all basically all audio intelligence features are by default turned off so once we did that the next thing that we want to include are the headers headers are going to help us authenticate ourselves with the authentication key that we have so let's set that up the first thing that we need of course is the authentication key and the second thing that we need is to tell assembly ai what kind of content we're sending its way and here we are sending a json format and this is all to start a transcription job so let's see what assembly ai what kind of response assembly ai sends to us and then we can understand what we need to do next all right so this is the result we get from assembly ai it includes some information that we need for example the id this is the id of the transcription job or audio intelligence job that has been started we also have some other information like the language model that is being used or what kind of features are turned on so as you can see sentiment analysis entity detection and everything else is turned off and what we want is the content safety so let's see content safety is turned on so that's what we wanted now using this id we can actually ask assembly ai when the transcription or audio intelligence job is done and get the results so for that i'm going to create a while loop inside this while loop what i want to do is ask assembly ai if transcription is done and if the answer is yes print results so to do that we are going to continuously have to ask assembly ai if the result is done so i would create a function where i do exactly that and to inside this function we are going to need to create a specific endpoint for our transcription job that endpoint is going to consist of the endpoint that we have already and a transcription id the transcription id we are going to get from the result of uh the submission of the transcription so let the one that we saw here this is the id so very simply it will just be response.json id all right this will be the polling endpoint and once we have the polling endpoint what we're going to do is again to use the request library and send a get request because we are not actually sending any information any data all we're doing is to ask for data and we are going to send it to the polling endpoint and i'm going to include the headers again for authentication of course as a result we are going to get a polling response this response is going to tell us whether the transcription is done or not you can already actually see it here if i look for status status is queued immediately after you send it it turns into processing and then after a while it is completed or if it's not completed if there was an error then it errors out and once we get the polling response we can actually return the polling response here with the information and i will call that function here actually of course this needs a transcription id and here i would need to call it with the transcription id and the resulting thing that we get i'll just call it data for now so just as we've seen here we have the status so here to see if the transcription is done or not i would have to see if data status is completed or not and if it has been completed then we can print the data so we're doing that here we'll be doing that too uh what else we want to include we also want to see if the transcription has errored out we want to warn the user about that so i'll just save the status as is error then we are going to print for example there was an error and we can also print the error section of our results just to kind of see what went wrong in there so that's basically it that's all we need to do just something that i want to fix here is instead of just printing it i actually want to use pretty print so that we can understand what's happening a little bit better because we're going to get a lot of information and also i would like to add some waiting time in between asking assembly ai if the transcription is done because otherwise we're just constantly asking and there is not really a good use for it so we'll just say time sleep 30 seconds and we can say before we sleep print waiting for 30 seconds so that's all we have to do actually one thing that i forgot is that we need a slash here to build the url properly and that's all that's all we need to do to ask send assembly ai the content moderation job and then wait for it and get the results so let's run this and see what the results are looking like we got the confirmation that the transcription job has been started we have an id for it and now we're going to probably wait a couple of minutes before we get the results all right our transcription is done but i've just realized that i forgot to stop the while loop here so i'll just need to add some breaks here once we get a result or one set error is out uh so and i i will just stop this one but we have the results so let's go through the results and see what they look like uh yeah so this is what we get from assembly ai as we also seen before we have the text of what is being said when there was something that that might need to be moderated the severity of the situation and what kind of thing this uh what kind of label or class that this text belongs to here we got for example health issues we got profanity uh we got health issues again so i guess this talk was a lot about health issues so we got a lot of it so basically um by using assembly ai's api here with the audio intelligence you get the information of where this sentences that might be problematic was talked uh on or what what it was and also when so you have the start and ending timestamps of when this part was talked about and that's basically it if you have any questions about how we build this project or how to use assembly ai's api don't forget to leave a comment in the description below and before you leave don't forget to go get your free assembly ai api token either using the link in the description or by going to assemblyai.com thanks for watching and i will see you in the next video [Music] you