Step 2: Filterbank Feature Extraction for Speech Recognition | Zipformer + Icefall Tutorial

Hello everyone. Uh in last video we have prepared the data set. We have prepared the manifest files. Uh in this video uh I'm going to show you how to uh extract the features uh filter bank features and then um how to use those in the training. So let's get started. Uh if you haven't seen the uh data preparation video, I recommend uh you watch that first and then uh come into this video. So here as you can see uh in last view we have prepared a script. Uh this is the script uh to prepare the manifest files and uh here are all the manifest files. Uh we have created test and uh train manifest uh recordings and supervisions. Uh these files uh we need to zip all of that uh files uh in z format. spotted. Uh we in in this terminal we need to go into that directory uh data and manifest and inside this we can use gzip and then uh star. So this will zip all the files. As you can see all these are zipped. Now um what we need to do is uh we need to go into the local folder. inside the local folder you'll find uh this compute app bank libri speech so you open the script and then uh scroll down so here I have already made the changes um if you see uh there are multiple sets you just uh need to comment out all of those and then put whatever name you have given um in my case I have given train and test these two sets the last part uh if you see um in the name the last uh part you need to give here and then prefix would be the prefix of the names uh in this case it is en English um then um this is it and one more thing you need to change this number of jobs if you if you are using a very uh low resource uh system then you can reduce this number um ideally by default it is 15 so I have reduced it to 10 and Uh you can uh this is pretty much it. Uh you don't need to change anything else in the script and then you can uh go outside this folder in the terminal. Sorry. And uh you can run the script by using Python 3. Okay. Sorry. Uh so here um you need to use uh the icefall docker image. The command is docker run and minus it. It will be an interactive and then use minus minus rm. So that um after uh completion of this process it will remove the container and after that you need to mount uh the volume. I will mount the whole volume uh here which is /media and uh then you need to give access GPUs all and uh you need to give shared memory uh little bit of uh little bit higher shared memory uh by giving shm size. Uh you can give 2 GB. 2 GB is enough. Uh after that you need to write the docker image name. Um so in this video I'm going to use uh the K2 FSA. Uh this is a pre-built docker image by Icefall guys. This is K2 FSA/ icefall and uh then the tag that is torch 1.13.0 0 uh cuda 11.6 and then uh you put bash at the end. So this is it. You run the command. Uh as you can see I'm in the in the terminal of this container whatever I have created now. Now you need to go into the same directory. You just copy copy this path and uh go into the directory and uh after that you can start start the script. So you can use python 3 and uh the script is in local compute lip speech. So you run the script it will start u the filter bank uh feature extraction as you can see it will use speed perturbation it will use two types of speed perturbation.9 and 1.1 so basically what is in.9 reduces the speed of the data to uh 9x and uh in 1.1 it uh increases the speed of the audios to 1.1x. So u it will triple your data if uh let's say if uh if you have 100 hours of data then uh after this speed partition you'll get uh 300 hours of data set. The process is uh completed uh all the features uh got extracted out. Uh so here if you uh see in the data folder one new fbank folder is there. If you go inside that folder you'll find uh uh two uh folders. one is train and other other one is test. So if you open those uh folders you'll find this uh uh features. So these are the filterbang features uh and uh these are the uh path basically path to all the features whatever are inside this uh folder. So these are uh the two main things which you need in the training. Um so this is the training data set uh preparation that is done and uh one more thing you need uh is to uh download the Musan data set uh that's a noise noisy data set uh that is needed for uh noise augmentation in the training process that happens on the fly. Uh so for that you can use lot uh if you if you go into the uh prepare.sh sh script you can uh find the musan part. So if you use this command let's say download musan. So if I use this command you need to give the data directory uh whatever the folder wherever the folder you want um data and then let me uh say raw. Okay. So uh if you do this you will uh the data will get downloaded in the raw folder. I have already downloaded it in the in the raw folder. Uh as you can see this is the tar file that you will get and then it will this same command will u extract out the audios uh into this musan folder. What you have to do after is that you need to extract out the features for uh for this data set. Um as you have seen in the local folder there is one more script compute fb bank musan. You just need to open that script and then uh uh you don't need to change anything here. All of this um are uh for the Musan data set only. Uh if you want you can change the number of jobs but since this data set is not uh very big you can um you can use this 15 uh number of jobs. Okay. So before starting that you need to create a manifest. Uh let me show you how to create the manifest in in the prepare.sh uh script itself. If you go scroll a little bit down you uh you will find uh this lot let's say prepare musan this what this will do is this will create the manifest files which we have done uh using our own script for the training data set. So this lot say uh will automatically uh create those manifest files for musan. Uh you just need to write this command. I'll show you the command. Let's say prepare musan and then the download download directory uh that is data / raw/ musa musan sorry and then after that uh the manifest file manifest folder that is data/manifest and then if you if you run the script uh then this will create all the uh manifest if you open manifest folder you'll find this music noise and speech the this is pretty much it and uh you can use the compute app bank script uh this is comput bank motusan and then uh if you run the same you don't need to change anything if you you you can run it uh directly local/mputee appbank musan and u you just need to run this script it will start extracting out the features for musan so after completion of the uh feature extraction of Musan data set. Uh if you go into the app bank folder, you'll find the similar uh similar uh folders like the train folder, manusan features and then all these features are there and then uh you can see the cuts also manusan cuts. This is pretty much it. You don't need to do anything else to train the model. After that you can start training of the model. Okay guys, this is uh this is it for this video. Um this video was about only for future extraction. Uh in the next video we will start uh the training process. Stay tuned.