Building projects with fastai - From Model Training to Deployment

Key Takeaways

everyone welcome back to the channel data science with harshit so as you already know that deep learning is inducing revolutions across many industries and we are going to talk about one of the libraries that is in the news these days and that is fast ai now fast air is basically a research institute and the founders basically uh jeremy howard and sylvia rachel thomas so these are the research scientists who are working uh at fastair and it is basically dedicated to make deep learning accessible uh to all individuals be it beginners intermediates or deep learning practitioners experts in the field and the goal of this particular video is to basically help you understand how easy it is to get started with the deep learning and build projects using fast ai we are going to make an image classified basically a dock classifier based on the breeds and the images that we download and we'll see how easy it is uh going from model training to deploying your application so it's it's totally uh you know broken down for us using the fast ai library and a bunch of notebooks that these guys have prepared that we can refer to as well so i have created this tutorial uh just to help you get started with the building projects so they'll this will be a series i'll be rolling out a lot of videos on fast ai as i personally find it very interesting uh it's using pie torch and yeah the only prerequisite for this is that you should know how to code in python that's all uh you should know uh program programming and especially in python so that's the only prerequisite and you need to know like high school mathematics they'll be covering elementary mathematics and high school mathematics that will be required to learn uh how those models are being trained mathematics is required it's not that it is uh it doesn't work that way but yeah mathematics is at the core of deep learning so that is required but yeah so without wasting any time let's dive right into it so before we get down to building the model and training it we need to import the required libraries and utility functions from the set of notebooks called fastbook now this fastbook is basically developed to cover the introduction to deep learning using fast ai and pytorch so this is a set of notebooks that is prepared and generated by the fast ai team now what we have to do is we have to firstly install fastbook in our environment which i have done that already and then we're going to invoke the setup book function uh which will basically prepare all the methods that we need for our uh program and our models to run to extract data and and a bunch of other uh utility functions that we're going to use throughout our notebook then uh the second thing that we have done is we have imported all the classes and function uh from fastbook so from facebook import asterisk that is importing everything that we have inside that package and then from the widgets uh from fasta so basically some uh gui related uh stuff that we have we'll be using for data cleaning uh later on in our tutorial so i've imported these required libraries run these two cells and and that'll be all so now we are ready to get started with our model building and the next step that we have is uh we for most types of projects uh you can find the data online from various data repositories and websites so to develop this doc classifier we need to have images of dogs and there are many images of dogs available on the internet and and to download these we have basically we'll be using uh bing image search api which is provided by uh microsoft azure uh cognitive services so you can sign up on microsoft azure uh and you will get uh credits worth two hundred dollars so again uh on microsoft azure you have to go to your portal and this would open up and i have created a resource group already so this is my resource group and here you'll go you'll click on free bing so this is my free server that's running and i have uh so if i'll go here on keys and end point uh from here i'll be able to copy my keys which i will use to query uh data from the bank image search api now uh i have copied my key from there and i will paste it here what we what i'm doing here is i'm simply uh setting my azure search key uh to this key that i have copied from here i'll just simply have to paste it here uh and that'll be all i'll be able to use the bing image search api now the fastbook library comes with several utility functions like search image bing that returns urls corresponding to several search queries so it gives you a bunch of things and we can learn about such functions using the help function so if you want to know about like what all functions classes are available in fastbook you can simply pass it in the help function and you'll see that we get a lot of information about what this uh library is about as you can see this is uh a package and you can see that we have a bunch of functions like cluster columns draw tree get image file sorted image bear then there's this function called search image bank which accepts key term and minimum size and there's a bunch of things that we can learn more about so the first step would be to gather data now in order to gather data we'll be using the bing uh image search api so i have this search image bing function so this is a function in fastbook and what it takes is the key that we have defined above uh which we copied from our uh portal on azure and i have passed the query this is my search query so basically i'm looking for german shepherd dogs uh the images of german shepherd dogs and then uh once you have passed it to this uh function it will give you some results and from those results we are interested in simply capturing the urls of all those images so that's why using the attribute got function so we passed on that we are interested in the contact url key from the reader that we have in results and there from there on we have all the images url urls of all the german shower dog images so that this will give you simply the images and we see that we have 150 images of german shepherd dogs as we saw uh we as we see basically we are checking the length now if we take a look at the images so this is you can see this is the list of 50 uh image urls and let's if you if you want to look at one of the image so we can do that as well but before that since we only have the urls as of now we want to download the images so uh for that we'll first define the path the destination path where we want to download these files so this is our test so i'll create i have an images folder and i'll create an example.jpg and what i'm doing is i am using the download url function this is again defined in the fastbook package and i have uh passed the this is my url image is one from the list that we have here and i have provided the path where i need to download it run it and now we can open that file as well so using simple image dot open uh the image uh is another uh class that is defined and we have image dot open pass the path and you can simply uh pass to underscore thumb 128 by 128 the size of the image so you see that this is how we can visualize the image or the downloaded image that we have uh in our system that we just downloaded from uh our bing search bing image search api so now that we know how to download data let's go ahead and gather our data for our doc classifier so i am going to work with the three different types of docs so i have created a list of types of dogs so i need german shepherd black simply black dogs and labrador which is another breed of dogs so i am i'm going to have three classes for my problem for my image classifier for dogs and then i have created a path basically a directory name dogs and inside of it this directory this is the main directory and inside of it i'll have three folders for each of the type of docs that we have in the list all right so let's go ahead and define this list now here what i'm doing is i will simply check if my path exists i will create a directory if it doesn't and and then again i am looping over each of each type of dog so german shepherd black and labrador so here is a loop that i have created i have first set the path i have created a destination i will check if that particular directory exists basically for let's say german shepherd i will check inside the docs directory if german shepherd director exists i will not create it if it doesn't i will create it so exist okay parameter basically tells us if it exists or not and then i will pass the query as well as the key that i have defined the bing image search api key and i will be searching for each type of dog trying to get the images from the bing search and then i would be downloading download images from the destination into the sorry to the destination and i will be getting the urls so i will be downloading the images from the urls that i will get from the results of the image search api and again attribute got as as we saw earlier how we download the images so again we can check for now uh the images are all downloaded in our path now the next thing that we have to do is we'll have to check if what these files are so we'll get these files so you can see we have 379 images that are downloaded and these are the images the path to the images these are the files we can check if a particular file is corrupt or not so again we have this verify images from fastbook we can check for any corrupt file so i've added this list of files to the verify images function this will check for any correct files and give us like the number and the path as well so here in our case we do not have any but in case you get any corrupt file you can delete them from the path you can remove all the failed images uh by mapping uh the unlink function so path dot unlink and it will delete all those corrupt files from your directory now we need a mechanism to provide data to our models and fast ai has this concept of data loaders that store multiple data loader objects passed to it and it makes them available as training and validation set now to convert this downloaded data into data loader object we have to provide four things now these four things are very important firstly you have to tell what kinds of data we are working with then you have to tell how to get the list of items that you will be working with and how will you label these items and lastly you will have to tell the model to this data order object that how to create a validation set from the entire data now to create these data loader objects along with this information that we have to provide now fast ai offers a very flexible system called the data block api now we can specify all the details of the data loader creation using the arguments and an array of transformation methods that the api offers so here you can see i have the data block class and i will be passing all of these attributes so firstly i have specified the block now this block basically specifies the feature variables that is images and the target variable which is the dependent variable that is the category for each of those images then i have get items now this get item retrieves underlying items which are images in our case and we have get images files function that returns a list of all the images in that path next we have splitter which splits the data as per the provided method i have used random splitter and uh we are basically randomly splitting with twenty percent of the data reserved for the validation set and specified the seed to get the same split on every run then i have get underscore y uh the get underscore y basically is the target variable is referred to as y to create the labels we are using the parent label function which gets the name of the folder where our images uh are actually residing so this will basically capture the name of the folder which would be the label so this is we have also provided how we are going to label now the last thing is item transformation that is how we it transform all the images since each of these images are of different size and the model accepts a file in batches so we need all of these images to be pre-pre-processed into a same same similar size so we are using the resize function and we are resizing all the images to 128 pixels so let's run this data loader object so this is basically the template we haven't done anything so uh we are going to again call the data loaders uh function from this dog's object that we have created so this is our data order object and i am invoking the data loaders function provided the actual path that we will be using to create the convert our data into the data block object so let's uh run that and now we can actually look at these data loaders so it has given us training as well as validation set so first let's look at a batch function a batch of validation set so this will simply visually help us visualize a few validation set images and then we can also look at a few uh training set images so this will take a few seconds and yeah you can see these are a few training set images that we have in our data loader object so the data loader is now uh ready to use uh what we need to do is we can add a few transformations uh transformation just to add some uh create more variation so we can basically add item transformation so we can use random resize crop with the the size that we want to resize it and we can specify the main scale now the min scale is the the most important parameter to pass which determines how much of the image to select at minimum uh each time so this is 0.3 basically 30 and then uh data loaders uh we create a data loader object again so this is just to when we want to add more transformations to the images again create the data loader object and then we can look at the batch itself we can see how we can uh add transformations to just one images so you can see we have resized each of these randomly resized each of these images and we have created crop so each time it's simply using it it is at least using a 30 or 0.3 the minimum scale of the image so that's how we can add transformation now data augmentation as we as some of you would know that we can add item transformations that is we are going to resize all of the images that is again fine now we can also add batch transformations that because the images would be going inside the bases now that all of the images are of the same size we can add batch transformation now there are standard uh transformations like which are covered in the augmented transforms function so this would include simply rotating flipping uh then changing the brightness or contrast a few other transformation functions and multi equals two means that we are also using uh double the amount of augmentation compared to the default and then again we create a data loaders object and now we are ready to look at a batch of training set so this is again we are simply loading eight images and two rows so you can see a bunch of transformation we could actually create a few variations of the images that we already have just to help us help our model uh generalize better uh looking at different sort of images so now it's time to train our model uh with these limited number of images so as you know we have just uh close to 370 380 images uh with us and we'll be training our model with this uh limited data set so fast ai offers many architectures to use from which makes it very easy to use transfer learning and we can create a convolutional neural network model using the pre-trained models that work for most of the applications or data sets so we are going to use the resnet architecture it is both fast and accurate for many data sets and problems so as you can see i have created a tran added some augmentation again a random resized crop item transformations so resize all the images to 224 pixels because that's the standard and it works for almost all sorts of image classification problem and then min scale is 0.5 i've added batch transformation this is the standard augmented transformation that i have added uh here in batch transformation and then i've created data loader object uh this is fine now here i am creating a cnn model you can see cnn underscore learner uh provided the data using the dls the data loaders object and then resnet18 now the 18 in this resnet 18 model basically represents the number of layers in the neural network so we also pass the metric so the metric here is error rate so this metric is to measure the quality of the model's predictions using the validation set from the data loader and we are using error rate which tells how frequently uh the model is making incorrect prediction so we want it to be as low as possible then we are calling we we're basically training so fine tune is basically analogous to fit method which is uh and which is available in other libraries like skeleton or transfer flow now to train the model we need to specify the number of times we want the model uh to look at each image so this is basically the number of epochs we want to train it for so i am training it for four epochs here so uh run it and this will take a few seconds oh sorry a few maybe take a few seconds if you have if you have a gpu but if you are running it over cpu that is fine as well this is simply uh this won't take a you know a long time but this would this might take uh like up to three to four minutes so as you can see we can actually look at the training loss validation loss error rate and the time it took for each epoch to run so you can see that the train loss started from point four eight in the first epoch went down to point three five then point two nine and then point two seven and the fourth epoch we can actually look at validation loss which was it started off as 0.58 a little higher uh because it had unseen data again uh this is obvious then error rate again uh kept on decreasing from 0.25 to 2 to 1 and then it remained the same and it took at least uh two minutes uh 15 seconds for each epoch to run all right so this was this was the model result so this is how our model performed we can actually uh look at the results by visualizing uh the predictions uh through a confusion matrix and we can do that using the classification interpretation module and here from this so we can use the from learner uh so basically in fasta and pytorch you are calling it a learner uh so the model is basically a learner itself so we are passing this model that we have created to this from learner function and this will plot the confusion method uh we are using the plot confusion metrics from the object that we have created from this class a classification interpretation uh class so again a few seconds and this will give us the confusion metric uh which would uh help us uh compare the actual values with the predictions so you can see uh we so the this is the main diagonal so this means that we got all of these predictions correctly and then these are the wrong predictions so we have three plus two five plus eleven uh so uh sixteen uh of the images were wrongly predicted right so we can actually take a look at these top losses so let's say we want to look at the top 10 images so okay so these are the top 10 images so you can see let's look at nine um or let's look at six in in three rows so it's giving us uh german shepherd so this is basically the prediction uh which is and then uh divided by slash then it's giving us the actual value then loss and then probability so this is how you can interpret these values so they're merging into one another but that's okay so we have german so this is basically uh predicted as german shepherd but it is actually black but i think that it would be both because it's both it's a black german shepherd so that's totally fine i took it i basically decided to look at or take up this data set with both black as well as because black would be labradors as well as you can see here as well so this it's kind of confusing for the model to understand so that's why i took this data set uh to show that we can also clean the data set so again uh fasta offers a handy gui in case you want to use these model training to clean the data set you can do that as well so we have image classifier cleaner you pass the model to it and it will basically uh open up a an ipi widget which using which you can actually uh change the classes the labels as well as you can decide like which one you want to which of these images you want to keep which one you want to delete and all of those things and you can then set them and then write a simple python command to delete or to change or move the images to the the directory if you want to uh clean the data set so let's look at uh how this loads up so as you can see uh we have got all the images which are which can be clean so these are all the wrongly predicted images from the training so you can check for which training set or validation set you want to clean which of the type of dog that you want to look at so you can decide you want to keep this image you want to keep delete this image you want to change uh the image uh or the label of this particular image you can decide which one it should go into and you can ch uh you know to make the changes for each of these images now if let's say i want to move this one to let's say a black uh you know uh to this is already black okay so let's move it to labrador so this is elaborate. so we can simply move this file to the labrador directory and here we can simply capture the index of this image as well as the category so using the change function from this cleaner object that we have created and then we can move it using the shuttle uh as a tutor the utility function from we captured this index the image and then we provide the new path where it needs to be copied along with underscore fixed so this would uh basically move with the this image along with adding this underscore fixed is basically because uh this index might already exist in the other library that's why i've added this suffix so this is how you can clean the data set uh by training your model now once you're happy with it once you're happy training your model you can actually export the model but that is only after a round of uh you know a couple of rounds of hyper parameter tuning that you would have to do uh unless you are you know until you are happy with your model performance after that uh in order to move from a jupyter notebook to deploying your model to a server you will have to save your model so we have export function this basically exports both the architecture of the model as well as the learned parameters so it saves both of those things and then we can save the path first define the path where you need to save this file and then this basically checks if there's any dot pkl file and then it will save it so we we can see that we have an export dot pkl file uh it's a pickle file that that's how this model is basically saved uh in a pickle file so that's the file extension basically now we can load the model from this pickle file uh use the load learner function and provide the path where you can load it from and now you can make predictions using this model inference so i've created this is a model dot predict is the function pass the image that you want to predict so this has correctly predicted as labrador this is a tensor okay so and it also gives you the predictions uh basically the probabilities for each of these classes as and you can see that uh for labrador it was the highest and this was it was very low for the other two classes you can also check for the vocabulary basically all the labels so we have black german shepherd and labrador the third index so that's why it's right here and again you can add interactive widgets to your notebook so that you can create an application out of it that's the whole purpose from jupyter notebooks from collecting your data model training to deploying your application now i have created a file upload button uh button to classify uh then there is output button and then labels so you'll see then i've created this function uh which will basically simply upload the file and then there's a classify button that will simply uh use the learn inference uh the model to predict the image and then i am printing basically i'm outputting the this is the label uh with the prediction and the probability for that particular class so let's run this and then i am adding a vertical box so this is again uh you can simply select the dog this is the message that i am printing out then this is the upload button then return button output and then label of the prediction so this is what i get so you can actually design as per your your preference and i have covered the entire voila series so again i will let's let's try this out so so what i'm going to do is i'm going to predict i'm going to upload a file so let's upload a file i'll upload a german shepherd dog so let's say this one now classify so you can see it has output this label the prediction is german shepherd probability is one so again this is a great result now in order to create a voila application you can simply click on voila or you know you have to you need to have voila installed and i have covered the how to create or how to generate a volan notebook out of your jupyter notebook and then you can deploy it you can use binder you can use binder to basically deploy your application it's very easy all you need to do is simply follow five to six steps you have to add your notebook to get a repository insert the url of that repo into binders this url field then you have to click file drop down to this url here you have to in this you have to mention how you are going to render so uh walla render and then name of your name of the ipython notebook and yeah that's it then you have to click a clipboard button at the bottom right to copy uh the url and paste it somewhere safe because that will be the url and then click on launch and your notebook would be live so check this out try it for yourself this is how easy it is uh with fast ai uh that you can go from collecting data to deploying your application in in just a single go so do try this out and let me know so i hope uh you all liked the video uh you all liked the tutorial and the concepts that we covered now i'll be releasing a lot of uh other projects based on you know built on fast ai and yeah if you liked it please give it a thumbs up subscribe to the channel help us grow and yeah watch out for the next video which is coming out soon and until then keep learning data science with hershey