OpenAI’s Whisper is AMAZING!

Key Takeaways

seriously is that enough to pull this off did you just saw the latest hot open source model released by open AI transcribing this guy it's called whisper and it's really really good well in reality what you just saw didn't happen in real time that was just me and this video but I did put together some code so you can see how good this model is and try it for yourself for context whisper is the speech recognition model that you can use for transcription and translation South Florida is one of the most beautiful places in the continental United States I'm gonna run that text through the model but we can make it even more interesting we'll see if this model can transcribe the audio and translate it into English so let's bring the computer and get started alright before we get started I want you to assume that the average human has a life expect expectancy of 80 years and sleeps 8 hours every single day that gives us a total of 467 000 hours where we are awake in contrast open AI used 680 000 hours of data to try and Whisper that is around 45 more listening time that we get in our entire lifetime so no wonder whisper is really really good now the model doesn't specialize in any particular task but open AI claims that it makes around 50 percent fewer errors across many different sample data sets that's just nuts final thing I'll say before I shut up and take a look at the code whisper is not only English which is huge about a third of the data set that openai used to train whisper is non-english so you can use the model to transcribe from a bunch of different languages all right let's look at the code which is surprisingly short by the way you'll find a link to this notebook in the description below alright so first I'm going to install whisper directly from their GitHub repo and radio and the reason I'm using radio is to create a very simple interface where we can record the audio directly from my computer and transcribe it and translate it that interface is based on a notebook that hugging face created and published online I took it simplified it a little bit added a couple more things and that's what you get here for the model itself open AI offers several options and I copied the table from their GitHub repo so here you have the list of different models that you can load on your computer depending on how much memory you want to use how fast you need the results to be or whether or not you need multilingual support personally I'm using the medium model but I found that the base model which is way smaller works very very well as well to start things off I loaded that model and then I created a couple of functions and I want you to notice how simple these functions are first there is a transcribe function that's going to receive a file and that's the audio file that's the recording and then we're gonna call the transcribe function of the whisper model that is here however that I'm passing a list of options and one of those options is the task that I want to do within my function and in this particular case the task is transcribing the audio the translate function is very simple as well and it's almost a match of the previous function except here the task is going to be translate now remember right now the whisper model only supports translating into English so you can start with any language and translate into English finally this is the gradual interface very very simple interface you can see it here it's got a couple the button want to transcribe my audio one to translate my audio and then you get here an area where we are going to display the text the result of that transcription or translation and the code is very straightforward you get the capturing of the audio here the component that's going to capture that audio here you have a couple of buttons the transcribe and the translate button and notice how I'm connecting the click event on these two lines I'm connecting the click event to the two functions that I created before therefore when you click on the transcribe button we're gonna call the transcribe function we're gonna pass the audio and we're gonna receive the result and display it in the text box that we added to the interface very simple stuff so this is everything we need for our example I added a final cell to my notebook where I'm calling directly the transcribe and translate function and this is useful if you want it for example to record your audio from your phone and then send it to your computer as a file you can upload the file and then access the functions directly pass in the file name so if you want to use that you have it there let's give this a try South Florida is one of the most beautiful places in the continental United States right so that's my audio South Florida is one of the most beautiful places in the continental United States sounds good let's click transcribe here and that was perfect that was fast that was beautiful all right let's try something else let's do it now in Spanish okay that sounds good let's transcribe it that was perfect that was very good and now let's translate it's gonna take the same audio translated into English the progress of Science in the last decade is incredible that was amazing okay so here you have it this morning I just saw people putting together samples where they are transcribing and translating YouTube videos music to get out the lyrics it's really really cool the community is coming together they're starting to build super cool things with this is open source so you can use it right away you really need to give it a try remember the link to this notebook is in the description below so how about it go nuts build something cool and I'll see you in the next one no no wait wait wait wait you made it all the way here so please like the video below subscribe to my channel and now for real I'll see you in the next one