Image Classification using Deep Learning Models | DataHour by Mohanraj Vengadachalam

Key Takeaways

hello everyone good evening um so I'll be talking about uh how to build an image classification model uh using ncnn architecture so before I go into detail of what is convolutional neural network uh so I'll few minutes I'll talk about what is the computer vision and then what are the different applications which can be applied for an image classification and then I'll jump to the various layers in CNN architectures and then how can we create a image classification model for a binary classification and multi-classifications I'll walk you through with the Hanson collab notebook file which already have a couple of links from my people so I'll be sharing you the notebook URL as well so you can practice it offline so close this section so this will be uh more uh um you know the objective of the sessions um so computer vision it is not a Nemo so uh basically most of us we have been using in our day-to-day uh you know in our life for example uh when I say uh like most of us when you try to take a pictures right it is automatically try to detect the faces present uh in in the focus of the camera right so most of us mobile comes with the inbuild the phase detection algorithm which is again a deep learning based model um to put a bounding boxes around the faces right so and the other examples which I can quote about um like uh when you upload a photograph into the social media website right like I can talk about face ID that automatically trying to attack the person's name right if there are persons names uh which is not tagged it will ask you to give a label for the particular person those images will be considered as a training data for the model in near future if you upload the similar person photographs which can be considered for three inferences are to identify the person's name so and the there are other examples which I can also quote um um like most of the matrimonial sites uh they are trying to use um a computer vision uh based approaches to detect the fake preference for example when someone is creating and profiles in the matrimonial side so it first they will try to have a filtering mechanism uh like identify whether we uploaded photo is a celebrity or not usually celebrities will not create a profiles in the management right and then they will try to estimate the age of the persons from the photograph and then they will try to compare from the descriptions of what the user has written so um these are the some of the examples which I'm quoting for the how CV computer vision like has been used many of the applications and and also you can talk about the snapshot filters like for example here when when you have a snapshot filters ready to try to show you uh emojis on top of our faces right for example based on the emotions of the person whether the person is happy or sad um there will be a emojis based on the uh the emotion expression classification it works for example uh to quote another example if you have purchased uh specs in the lenskart website right nowadays you don't necessarily to go into the team person to the store and then via respects and then you feel uh you know how the specs it looks how it fit for the each of the persons right and nowadays if you go to the website um they have a facial Landmark points um it will try to detect the facial animal points and then they use augmented reality like whatever specs you choose it will give you an output image with the augmented uh like uh the space specs will be overlaid on top of the uh the recorded faces whatever the input you implements there are a couple of other use cases we can keep talking about it but uh just to mention few of the application where the computer vision has been used um so what is mean by computer vision so uh like what are the difference between the human vision and the uh computer vision so um uh like when I say a human Mission uh uh through our human eye it captures uh and and then we have objects within the bound and then our brain um as the knowledge pattern of all these objects for example whenever I meet some someone uh new right we introduce each other and and then like capture uh the patterns or the unique person's pattern and then I I'll have the knowledge pattern of the particular person and then I say the person x y e r is that right so now like uh the main part here the human brain so uh in terms of computer vision the human brain will be replaced with the machines and the camera will be replaced with the sensing devices so I will be replacing the sensitive devices and only the difference here Nation to perform some analytics on the video file are the images you need a two browser there will be a timing and then inferencing part of the testing part um the training will happen on the offline process where you will have a larger number of sample uh you will give a larger number of samples for each of the unique objects which is present here or which you want to perform a classification are categorized and then you trying to model a base that is the training data samples uh the trained model what we call as the knowledge patterns okay so the once you have the trained model images or the unknown sample model will be able to categorize what that object has present or what that object belongs to so um but here we'll not have a concept called as a model training but uh even like how uh the newborn babies uh how we are trying to um you know teach our babies to understand each of these objects right similarly we have to make our missions as an human brain uh that we need a larger number of samples mostly the Deep learning approaches are the data-driven approaches where you need to provide a larger samples to try in our model based on the kind model we'll be able to categorize uh like based on our number of categories so we have a two kind of a classification problem one we'll talk about binary classification we say true or false for example uh person or not or a cat or not so these are the uh binary classification problem when I talk about multi-classification problems when you have a more than one category uh or more than one classes in your data sample in your data set then we'll go for a multi-classifications for example we'll talk about uh amnest the handwritten digit recognitions so we have a handwritten digit uh between zero to nine so there are 10 different unique uh handwritten digits so we call it the multi-classifications um now let me go in detail of uh how we can build a model uh for the Android and digit recognition so um in terms of computer vision uh uh you can solve various problems one of the problems are the image classifications um like uh uh in this scenario I have an object where my model is trying to categorize what are the object which is present so there are multiple categories present or multiple class labels are present that my model is trying to predict the bottle Cube and cup okay but in terms of classifications one of the limitation is that it will give you what are the objects is present within the frame or limited the images but it does not give you uh what pixel region or the pixel coordinates the object is present for example um you are trying there are multiple persons are there okay multiple objects are there we it does not give you which pixel coordinate uh is the curve or which pixel coordinate is the cube or if you need the uh the boundary box coordinates as well um we have the X mean y mean and the x max y Max we call it the uh the rectangular or the bounding box coordinate uh you if you want to know the uh each object the pixel coordinates then we go for an object prediction or the localization problem and you'll have additional informations um about the each of the object which is present in the images then we have a problem called of segmentation we talk about semantic segmentation and instant segmentation in terms of instance segmentation uh when you refer object detections right when you put a bounding boxes for each of the objects present in an images you will see there are few pixels regions which is not a part of the objects right so uh what does the instance implementation is right this like from the detected object it is trying to generate a mask it will only try to extract the exact pixel coordinate to each and of the object which is present um so uh that is the beauty of the instance segmentation uh coming back to a semantic segmentation uh it is a kind of a pixel level classification okay um so uh what we'll do here um most of the architectures we call it as a segment and the unit um especially for the autonomous vehicles many deep learning architectures uh related to semantic segmentation as we use [Music] at a pixel level uh we'll try to happen at the spatial domain across the uh all the uh Matrix but here we will try to do an individual pixel level we will try to do a classification um if if the prediction belongs to the background then we have Ascend as a here it is visualized but in terms of instance segmentation doesn't bother about like whether it's a unique object individual object for example there are three cubes are there each tuber assigned with a unique color but here in terms of semantic segmentation um we are trying to prove the uh objects which are which belongs to a similar classes will be assigned with a unique color value uh so um in terms of example to show you some examples for example we are trying to do a collusion detections in the road scenario okay um he wanted to detect what are the vehicles in front of uh the uh moving vehicle um for example here I'm trying to build a object detection model where it is trying to detect the traffic sign and the traffic light and the persons and the vehicles so uh you will be able to detect at what distance the vehicle is present so that we will get an alert at um [Music] [Music] you know the road scenario which can be a possible one of the uh use case I'll pick it up the question and answering session at the end is that okay right so I'll spend my 10 minutes of my effort at the last attention it's fine right and uh in terms of image most of the computer vision image and video we'll talk about images right as a human being how we perceive an image is um uh the eight for example here um um I have an image 8 which is represented so how we perceive an human being um but in terms of nation representation of images okay and here the one which I have represented is a glacial images usually it will be in images a you will have a number of rows and the number of columns we will say the pixel resolution usually it will be a like and tricks will be a which will be a larger one but here um to make you to understand I've just given you a smaller Matrix which is 18 plus 18. um uh if you look at here the pixel 0 represents the black color and 255 represent the white color then we'll have a great shades of values between 0 to 25 the one uh the representation of image pixel coordinate which belongs to dgd will be a foreground the other the pixel coordinate most of the pixel coordinate value which belongs to zero will be in background pixels okay um so uh at the end our machine is going to perceive and images it will be a numbers uh range between 0 to 255 so in this case I have 18 class 18 the total number of pixels will be a 324 pixels each pixels will be a one numerical value so there are 324 numerical values which I which we can talk about uh but uh if it is going to be in color images then usually I'm just representing so which will be a color Channel images but in terms of basically the depth will be one but in terms of the color images the depth will be three um uh so azim I'm I have a data set um a binary classification data State uh which I wanted to build a model uh the use case here the applications uh dgt8 and the not digital okay um whenever you are trying to build a model make sure that the data happens the diversity with the diversity of data which is possible uh in your production environment or your test environment okay uh if you are a data sample of your training data sample is too biased towards a particular environment and if your model is deployed in some other environment it it may not work well so um so when when you do a data collection for example in this scenario where I'm trying to build a model uh to categorize whether the handwritten digit is eight or not which is a binary so 8 will be a true scenario or not a it will be a negative scenario um so in this each person the handwriting style will be different okay and someone will be a rotated style someone will be a tilted position it will be a smaller one so um make sure your data samples covers all the scenario which is expected in a production environment so try to collect your data sample so uh in the true scenario you will have a image sample which has only the digital but in activity scenario you will have other than digit 8 will be a negative scenario so since I have thousands of sample images which and thousands of sample images which belongs to a non-digitate okay so now I'm going to build a deep neural network model um which the which is going to uh perform um binary classification of whether the given input image is a digitally Okay so so now usually when I say a deep learning uh we'll have a input layer and then the hidden layers and the output layer okay um so the input layer will consist of a n number of nodes so in this case uh I have created a input node with the 324s okay the shape of input a layer shape will be 3.4 since uh my input image Dimensions will be 18 cross 18 which which reshaped into one one dimensional Vector of 324 okay so now this 324 each node will be a one pixel intensity value there will be a 324 uh node uh each node will be a one pixel intensity value which is going to be a input to the hidden layers okay all this is [Music] foreign [Music] [Laughter] [Music] [Music] um so I talked about um like how we represented images and then um to talk about various layers in the Deep neural network so in terms of input layer we'll have a number of nodes will be defined The Escape of uh input nodes will be defined if it is going to be an images [Music] so I'm going to have a number of input layer nodes will be a 324 um which is given as an input would be hidden religious so here two hidden layers the problem is persisting none of them can hear anything properly did um [Music] so once we have a hidden layers created then we'll want to Output layer nodes if it is going to be in binary classification so we'll have a single node at the output layer where we will get a probability score between 0 to 1 and then uh [Music] to categorize uh like we will put a 0.5 probability score and negative scenario so we'll get a probability score at the output layer nodes and then we'll try to take a business Logic No threshold and to categorize whether it's a negative classes are the positive classes so this is how it works um so it will have a two kind of feed forward and the backward provocations uh during the feed forward Network the hidden layers will have a set of Weights basically going to work in terms of deep learning architectures and let's assume you have some business problem or use case which you want to come up with the solutions of a deep learning or a computer vision based Solutions considering I have a problem statements uh which I wanted to uh which is basically whether the person is wearing a face mask okay so this is one of the a business which you want to build the solutions with respect to deep learning the problem statements then the next thing uh is to a data accusations are the data collections part um so here when I say a person bearing and face mask like uh maybe a different genders a different age group a different regions for example Asians or the face the skin tone might vary and the different face mask colors so these are the some of the factors we might need to consider when you and another part is that your data should not be biased towards one single category of them which means unbalanced data might also create a model overfitting uh scenarios like unbalanced in the sense for example you have a two classes like a negative scenario and the positive scenario for example you might have too many samples for the a negative scenario where person is not wearing a face mask okay let's say you have you got around 20 samples uh for the person is not wearing a face mask but with the person wearing a face mask you might have only a thousand to uh one 1K to 2K samples okay in such cases the model will have a better understanding the knowledge patterns of the maybe your model biased towards the uh negative scenario than the positive scenario so so you might need to see uh how you can get a balanced data samples between across different classes in your data sets so this this is one of the uh you know factors you might need to consider when you try to build the solutions with respect to deep learning okay the another part is that um uh we need to have a there are multiple you can split your data set into multiple sets we can we can talk about um uh trying test and the validation set some people will talk about out of sample hold outside so it depends on the person to person but majorly we can categorize into a train and test okay um so uh usually we can talk about 80 percent or eighty percent of data samples for example if you have a 10 10K samples including uh both person wearing and face mask not wearing a face mask a 10K sample you can consider um 20 which means a 2K sample for a model uh benchmarking or a testing purpose remaining APK samples you can use it for the model train Okay the reason for data splitter um uh see once you build your model you need some unknown sample to validate uh the trained model right so we can't use the same set of sample for model training and testing so we should have a newer samples are the Unseen samples which is which you have used for a model training so we call this the test set so we can we you can follow different ratio 80 20 percent or an ID 10 even 70 30. but what some proves is that a larger number of samples has to be used for a model training than the testing so that is the thumb rule we should follow when you do a data the next part we talk about the feature engineering so feature engineering is nothing but like um when you have some visual images right I told you I spoke about the image will be represented in a matrix representation which consists of a numerical value right all these images uh are based itself a numerical value so all this pixel intensity might change with respect to several conditions like if the person rotate an images the pixel intensity might move to some other pixel coordinate if the illumination get changes for example if the picture is taken in a proper lighting condition or outdoor environment or indoor environment so the same object a pixel intensity might change if you zoom to for example if you are taking image which is very closer to the camera the image will be scaled out if the if you take a picture which is a larger then the picture will look smaller and and for example if you consider CCTV camera which is mounted on the wall or a 30 degree angle the object might be tilted or rotated so you will not have a frontal positions of the objects right so there are different factors the pixel intensity is not going to be in Statics which might get changed uh various scenarios so uh what we need to do we have to know instead of giving your raw data to the model we have to do a feature engineering um in in terms of deep learning uh this feature engineerings are taken care by the model itself but in traditional approaches if you consider an ml based approaches we need to do a handcrafted feature engineering you have to convert your image into an n-dimensional numerical value that represent uh maybe a color feature or it will be texture features or it may be a key point descriptors or it may be a histogram features um so these are what we call it's an undefined features and grafted features where the developer or user needs to Define what feature engineering has to be extracted one of the limitations with respect to handcrafted features uh it is an application dependent so I can't we use the same feature engineering patterns for the other application for example if I use a color as the one of the features to categorize the visual images I can't use the same color features to distinguish or to identify the face images so that are application dependent especially with respect to and grafted features okay so once you get a feature engineering at the feature pattern then we pass on to a a model training where we can talk about um [Music] um no it may be a supervised machine learning approaches or it may be unsupervised learning approaches rxpv neural networks okay then uh the last one will be once you have a trained model then we'll have a benchmarking of our model where we'll talk about uh confusion Matrix our applications are the recall it might vary the metrics uh uh my LED uh problems to problems so in this scenario which I have taken a classification problem which I have given here the confusion Matrix this is one of the Matrix note to measure how your model is performing and what will be the Precision and the recall based on the configuration Matrix we can come up with the Precision and the record um uh so now uh let me move on to the uh what is the Deep learning uh so basically uh deep learning when I say it is a deep learning architectures uh where you'll have a more number of layers which is stacked one after the others um so when we give the input images uh to the Deep learning layers where the first layer might be trying to extract the uh shape features are the Arizona line are the vertical lines um we'll use we'll call these features are the very basic features are the low level features use these lines are the edges to detect the shape of the uh the small component or the part of the objects and then pass this each of these layer output will be passed on to the uh the next layer in the architectures and then finally uh the last layer what you get is in high level features we can we can say the part of the object are the shape of an object and then it will be passed on to the fully connected layer at the hidden layers and then finally we will try to prediction output so we will try to see what is going to happen on each of these choir boxes uh I'll a little bit I'll talk about um what is going to happen inside these layers okay ah so the neural networks are the base for the neural networks will be our uh like it was derived from our human brain system where we'll say uh there will be a node or the neurons which receives the N number of input connections um and it does some computations um basically we'll do a DOT product between the uh the input vector and the weight vector and then the dot product some of the value will be put onto this node value and this is an extra layer in the network okay um so I as I told you like uh in deep learning we'll have a input layer and the hidden layers and the output layer uh I spoke about how do I Define the number of nodes for the input layer which is going to be an images then the each node value will be the pixel intensity and the output layer will be based on the what kind of classifications problem but in terms of hidden layers we'll we'll we need to come up with the hyper parameters where we have to um you know initially you can create two hidden layers with some number of nodes for each of this layer and then you can do a benchmarking on the test set and the drain set accuracy and then you can further fine tune those parameters um so when I when I talked about uh each node release the pixel intensity value it uh if not alone is the node value node value to the next layer in the network we'll also assign a weight value um so this weight value will try to do a model training so initially these weight values are get initialized with respect to a random initialization or you can use a gaussian distributions or some kind of a normal distribution so you can use some kind of a distributions to initialize this weight value each of these distributions has some range of values to initialize this weight value for example this blue color node receives two input connections so where we do a DOT product uh sum the value of this two node value and the Sumter value will be put onto an activation function uh especially uh the neural networks are used for uh like to get an non-linearity so to get a non-linearity uh into the neural networks we will try to apply the activation function to the neural networks uh so there are several activation functions so I spoke about hidden layers and the output layers but the input layer will not do any computations on the input layer just giving a node value to the next layer in the network so uh for the here we will be using a relu activation function most of the scenarios and output layer will be using a sigmoid and soft Max [Music] for the binary classification and damage we can use it for the hidden layers but in terms of computation and the accuracy wise there will be a lot of exponential functions with respect to our damage and the range of values between minus one to one plus one for the damage and the relu will be the output value of 0 to 1 and the sigmoid value will be zero to one so there are other activation functions as well so we have a parametrical leaky relu and then elu so there are other activation functions as well which I'm not going to focus to detail on this um uh so what is going to happen uh uh in the uh neural network hidden layers for example here I have a smaller neural network architectures uh hidden layer with a input layer with three nodes three values N1 and al2 and M3 N4 a kitten layer one with two nodes and output layer with one nodes okay assume it's going to be in binary classification um so what is going to happen with respect to neural network um each layer will be connected from I to I plus 1 for example which is connected to the both N4 and the N5 nodes okay so the total number of connections for the hidden layer will be number of nodes into a hidden layer into the previous layer which is the three so there will be six connection N4 will receive a three input connection and N5 is also receives three input connections okay but what what is going to make a difference here um the N1 um the N4 will receive a different set of weight value and the N5 will receive a different set of weight value for example N1 into w14 plus N2 into w24 M3 into w34 so you will do a DOT product uh between the each of the input node value along with the respective weights okay this weight values uh will be uh initialized um with respect to a random initialization we will try to learn this very value through a back provocations based on the error or the last value okay so similarly uh we will try to do a we will try to compute the N5 node value where we can talk about N1 into w15 N2 to w25 and N3 into w35 so uh you will get the dot product sum the value uh that will be passed onto an activation function so here the F of value represents an activation function the activation function output will be assigned to the particular node okay so once you've got the node value for n for an N5 then this will be acting as an input to the the next layer which is an output layer with the node value of n6 okay so now I I'll be able to compute the n6 value N4 into W 4 6 n 5 m 2 w 5 6 where you will get the dot product sum the value and the activation functions output okay so this will be let's urgent this will be a predicted value of your model uh so whatever input data we have given there will be an actual uh value uh will be there which we'll call it the ground truth or the label value let's assume [Music] error value uh I'll pass it here any questions uh maybe let me answer we need to take an image data set okay see if it's going to be there are two things out there um whether you are going to use a flying deep neural network RC and an architecture if it is going to be in CNN architectures um then we'll have a few more layers we'll talk about a convolution layer and a Max cooling layer and then we'll have a fully connected layer as the hidden layers but if it is going to be applying deep neural network either you can have a three to four uh layers in a network Market if it is going to be in CNN based uh deep learning archanges then it may be even people resonate architectures which comes with the 150 to 50 layers um see the depth of the layers what kind of GPU machines you are using to train your model and how many data samples you have so um if you're going to increase the depth of the layer then the amount of time is going to take for model training is also going to increase so it's not along the factor of increasing the layer and the volumes of samples and the uh Hardware oscillations uh to train your models okay um let's see uh Mission Vision we can we can talk about video analytics as well so I'm not going to talk about the we can also say as a computer vision um so video uh video analytic you're trying to identify some abnormality uh when someone is um in a video we can also perform a video classification that is all the kind of problems we can talk about with respect to machine it depends what what kind of industry applications whether it's automative industry or a medical industry or it may be a no Finance industry so it it varies with respect to uh industry to Industry so uh maybe I'll move on to the CNN based model uh then uh I'll I'll jump to the uh hands and how to create a deep learning based architectures then probably I'll pick a few more questions of them um so I I just showed you a simple deep neural neural network where we can have a input layer and the few set of hidden layers and followed by an output layer right when you when you talk about applying deep neural network or uh the client deep neural network does not do any feature engineering uh just it is trying to perform a classification alone one of the beauty of CNN based architecture architecture which can perform both a feature engineering as well as the uh the classifications [Music] uh a handcrafted feature engineering method you should have a separate block or a stage to do a feature engineering taking care of the feature engineering and classification will be put together a single architecture or a single block but in traditional ways are the approaches where you need to have a separately for the future engineering part um in in let's to show you simple uh sample architectures for the CNN so we'll have a layer called as a convolution layer and then booling layer and and then we'll have a hidden layers and the output layer the right side part which is already taken care in the normal new deep learning architectures or the neural networks but the new things which we are going to add and to the CNN architectures are the convolution um what is going to happen with respect to a convolution convolution is not a new term uh we have used in traditional image processing operations for example if I want to detect the edges okay so we use some kind of uh Sobel kernel or the LaPlace and Cardinal you will have a smaller Matrix where you will try to slide that smaller Matrix to get a feature patterns of the edges okay are the corners are the shape of an images okay but in traditional approaches all the filter value of the kernel values are predefined but in terms of deep learning based approaches we are not going to have a defined kernel but this will be learned when you do a model training okay so we will not have a only one feature in a single player for example if I say um so this I can say this particular layer will be a one convolution layer in a single convolution layer you will have a n number of filters okay for example here I have a five different filter each filter is going to produce a one set of features so you will have a five different features one square map represent the one feature map so in depth will be a number of features now so you'll have a n number of features now which is going to be passed on to the uh uh pulling layer or you can also have a another convolution layer uh we'll have a multiple convolution layer a Max cooling combinations okay so in this case I have a convolution Max pooling convolution and Max pooling and convolution and Max pooling one of the beauty uh having a consecutive convolution and Max cooling layer where you try to have a uh no high dimensional feature for example the first coin resolution might be trying to extract the uh like different parts for example eyes and ears and different parts of the cat and then you are trying to combine that feature at the last so where you will try to get the uh like uh parts of an objects are the uh shape of an objects okay so we'll try to have a multiple combination of such a convolution and the max pooling layer uh I'll show you in one of the example uh uh what is going to happen with respect to an image convolution so let me move to a URL um so we have an image here the same image is represented in a matrix representation and then what I'm going to do here um so I I told you uh we'll have a smaller Matrix okay so this is a smaller Matrix which is going to detect the edges in the images the sharp pen filters you can you can talk about different image processing kernels here Sobel kernel are the um uh if it is an identity it is the same matrices but I'm going to do the Sobel kernel uh so this will be the filter value so what is going to happen here uh you will have this 3 cross 3 or the Phi cross Pi Matrix this kernel will be a smaller Dimension which will be an odd size 3 cross 3 or a 5 cross 5 you will try to fit these three cross three and a 5 cross 5 kernel in each of the pixel intensity value when you fit this kernel you will see a pixel intensity value in a square boxes and the below after you will see a kernel value so you will have this 9 pixel value and the nine kernel value this dot product sum the value will be put on to the output Matrix or we'll call it the featured Matrix okay so whatever you see on the right side that will be a feature map so whenever you try to slide this kernel on each of the pixel coordinate you will see a 9 pixel value and the nine kernel value this will be a DOT product summed up value which will be put onto the uh you'll have the same Dimension as an output Matrix that will be uh example here I'll show you one live examples here okay so first I'll show you let me show you an uh original images of this so now this is the original image so uh I'm going to do a Sobel kernel so this will be the Sobel karna after applying the Sobel kernel to the original images you will it will the beauty of the convolution layer will be it will try to scan the feature patterns for example here it is time to extract the the gradients or the shape um it ignore the uh images the structural patterns of the images it is trying to extract so that is the beauty of the image uh you know convolution layer basically so I I showed you with one kernel how we are trying to slide this kernel uh across the uh each and every x y coordinate in the images let me go back to the slides uh so I told you we'll have a n number of kernels so we have a two kernel A and B uh with the dimension of Three cross three uh so when when I fit into the uh kernel uh so make sure you fit the kernel into center of the uh kernel into each and each and every x y coordinate uh so when you when you try to fit the kernel uh there is a problem of no uh so when you when you try to fit your kernel for the boundary of your your images right uh which will which will go outside of uh your pixel coordinate for example I'm trying to fit my kernel Center of Kernel to I i1 pixels okay so this scenario what happens here there are few pixel region which resides outside my uh boundary of my images right so in such cases where will not be able to do a feature scanning or the feature patterns that the uh the corner of the pixels are the uh first row last row and the First Column last column we will not be able to do a convolutions are done so we have another solution called the zero padding where we'll do a zero padding and the uh boundary of an images then we will try to do a image convolution with zero padding and without zero padding so we have a logic so um assuming uh we are going to fit into I6 pixel so you will have i1 into A1 and I2 into A2 so 9 pixel value and the nine kernel value and then uh the dot product sum the value will be put under output field management so there are two feature Matrix can considering there are two different Cardinals okay so I spoke about how do I solve extracting the feature patterns on the corner of the pixels as well so we we go with the zero padding so when you do a zero padding so only consider non-zero pixel value alone and do a DOT product and that will be put under output feature now um so to show you one simple example here let's assume so I have a smaller Matrix where I am trying to slide this three cross three kernel um sliding this kernel on each and every pixel coordinate from the left to right till I reach the last pixels in the images okay so uh the red color ones are the kernel value and the black ones are the the pixel intensity so after you do a DOT product sum the value the one you see is a convolved feature value which will be a output feature map um so we have a another layer called as a Max cooling layer in CNN uh so when you do a feature engineering with respect to convolution layer where you will get a multiple feature map at a single layer okay then the feature Dimension is going to increase so how do I reduce the number of each other dimension okay uh so the cooling layer will be applied next to the uh convolution layer we can have we cannot insert or we can't have a convolution layer um like Max pooling layer as a first layer so it it's supposed to be a Max pooling layer after the convolution layer so we have a two kind of a cooling layer we can talk about a Max cooling layer and the average cooling layer but it doesn't do any feature engineering especially convolution layer helps in extracting the feature pattern but in terms of Max cooling layer and the average pulling layer it is trying to reduce the uh the feature dimension in terms of spatial Dimension it will try to reduce the height and width of each feature that up but the number of depth will not change the number of feature map will not change and independently each feature map we are trying to apply the pooling layer of them so what is going to happen in the max cooling layer let's say this this Matrix is one of the feature map here uh so I have a two plus two as a kernel for the max pooling layer when I say 2 plus 2 there will be four pixel value or before feature map within these four feature map which one is having an IR value that will be a output value so and then uh what is going to happen here then you try to apply the kernel uh without overlapping so the first region I have applied the kernel and the second region which I am applying without overlapping but in terms of convolution layer when you slide this kernel we'll try to do a overlapping way but in terms of match cooling way will not uh most recommended approaches will be without overlapping when you slide your kernels okay so within these four pixels we'll have a maximum value and similarly you will come to then the next regions will be this particular region and try to find which one is having a maximum pixel image if they are the feature value so um so usually what will happen after uh you completed the pooling operation the feature Dimension usually will get reduced by the half of the dimensions so the input feature map will be a four class four which got reduced into two cross two uh the difference here between the average Cooling and the max pooling um in average cooling we'll take a mean value but in max volume will which one is uh within the kernel or the filter which one is having a maximum value so we'll try to do a pixel test or we uh try to find out which one is the maximum pixel intensity value within the kernel or the filter with respect to a pooling operation but in terms of convolution operation we try to do a uh we'll have a kernel value filter value and the pixel value you will do a DOT product uh between kernel or the pixel value but here you will do a pixel test within the input feature value so [Music] um I'll I'm sharing the uh um uh URL so uh you can use this URL later on so um so one of the advantage of Google just I'll give a highlight about the Google call up so uh it is a pre-installed all the libraries for example tensorflow carers you open CB all all these are presetter and also it will it will give you a GPU environment to run your notebook file uh what are the limitations here uh we can't run a larger data samples here and you can't have a larger depth of the layers as well so there are certain limitations with respect to a Google caller there are some pro versions also available so you can use it um so here what I'm going to do I'm going to use the uh inbuilt data set from the Keras Library which is a mnist handwritten digit data set so um so here I'm trying to use the Keras and tensorflow Library um so and I'm trying to import the necessary uh functions inside the Keras we have a hidden layers and then we have a convolution and Max pooling layer and then we have a mnist dataset so and the number of uh objects here I'm going to perform my multi-classifications problem and then I'm going to Define what will be the height and width of the images so the image Dimensions will be going to be a 28 plus 28 into one um so there will be a 60 000 data samples for the model training and ten thousand data samples for the models and each image Dimensions will be concerned uh into one so 28 uh will be height and width and one represents it's a grayscale images okay and here what I'm trying to do um now when you feed your data sample sample image data set into a deep learning architectures better to normalize uh the pixel intensity value so the pixel intensity values range between uh zero to 255 but here I am trying to normalize the pixel values into zero to one okay and and then what we are trying to do uh so we'll have a Target variable value which I am trying to convert the uh the target values into uh one not including representation are the binary representation um so we have a digit uh the ground root value between 0 to 9 uh so which we are trying to represent in a a 10 bit binary representation since we have a 10 different classes so this particular digit will be 0 1 2 2 5 6 7 8 9 so so in this case at this particular digit uh ground truth value will be six 6 will be represented in a 10 bit representation from this 10 bit representation any one position will be one the remaining all other Position will be zero so so all the uh ground truth value will be converted into one now one notification [Music] so we we you can also talk about period similar functionality we are trying to do uh for the ground truth value for your all your training data samples okay so now what I'm going to do and I'm trying to create a sample CNN architectures to perform a handwritten digit classification so uh first thing you need to create a object with the sequential um so we are going to add each of these layer to the uh object called Model so the first layer which I'm trying to add a convolution layer how many filters we are going to have I told you there will be a multiple feature now so how many featured map you need that will be a number of filters what will be the filter size so I I shared you I I showed you a sample kernel uh with the dimension of three class three right so so this will be a sample kernel so we can also have a five cross five as a kernel size so we need to Define what type of Kernel Dimension we need to have and what will be activation function so whether you need to have a relu activation function or a damage activation function so mostly for the hidden layer and the convolution layer we'll be using and reload activation function and here the next layer which I'm also creating a revolution is with 64 different filters but for the first convolution layer the input Dimensions will be at 28 cross 28 into one so the second convolution layer the output whatever feature map output we'll get from the first convolution layer that will be acting as an input to the next layer so for the second convolution layer I am going to use a 64 different filters and filter size will be 3 plus 3 then I am trying to have a cooling uh layer uh the filter size for the cooling layer will be a 2 cross 2 before we feed into the hidden layers we need to reshape uh the max pooling output into one dimensional Vector which will be flattened into one hidden nodes in the hidden layers and then uh the the last layer which I am creating an output layer with the number of classes so in this case it's going to be a multi classifications so I'll have a digits between 0 to 9 so number of categories will be attained uh so for the output layer alone we have a two kind of activation functions for the classification sigmoid and soft max if it is going to be an um multi classifications we'll be using a soft Max activation function if it is going to be in binary classifications we'll use a sigma and activation function so uh we have created a sample uh CNN architecture and then if you put a model that summary function so when you create a layers you have to use a model that had each of the layers you can layer we are trying to create and then if you put a model at summary which will give you uh what will be the output shape from the each of the features okay for the first convolution layer the input will be 28 cross 28 into one after you do a convolution layer with 3 cross 3 with 32 different filters if you apply one three cross three you will get a twenty one twenty twenty six class 26 there are 32 different filters so the depth is going to be 32 so you will get a 32 feature map with the dimension of 26 cross 26 with the square so the depth will be a 32. and the next convolution layer will be a again 64 different filters uh with the dimension of three class three so you will get a 64 different feature map after you do a convolution it will be a 24 cross 24 so uh similarly you'll you'll have a Max pooling layer which will reduce your feature map height and width of your feature map but the depth will not change so if usually uh if you use a 2 cross 2 as a kernel and the style value of 2 it will reduce half of your feature Dimension which is the height and width the depth remains a after you flatten then it will comes to a hidden layer required and the output layer no so now comes to the uh the backward publication so I I spoke about a model uh feed forward and backward propagation so when you do a backward replication so what will be the last mechanism so usually we use cross entropy last um for the classification problem if you go for a um like a pretty regressions or the predictive analysis you will go for a rmsc or the mean square error and here the weight optimization techniques we call it an Optimizer we will be using an atom Optimizer which is a gradient descent and Matrix I need in terms of accuracy so now um I'm going to feed in all my training data samples this will be the pixel intensity value and this will be the ground rule and what will be the path size so what is the purpose of batch here in this case I have a larger sample which is a 60k around 60 to 70k sample I can't do a feed forward for all the data samples and then do a backward provocations okay your RAM will get crashed so usually what what will happen we'll segment our data samples in a batch wise let's say I'm going to take a first 50 samples do a feed forward and backward verification then take a second a 50 sample so we'll we'll segment our data samples in a batch mode and then epochs will be a uh like how many times you want to do a feed forward and backward publication so this is also one of the hyper parameters um uh usually what will happen so we'll have we can set this effort to a larger Epoch then we can have a early stopping condition so there are early stopping condition which will monitor your validation loss and your training loss or validation accuracy then if your validation accuracy or the training accuracy or the validation loss it's not going to be changed for a continuous 10-year box then you can uh you can terminate your model training so there are a lot of things are there with respect to early stopping in the model checkpoint so even you can save your uh a trained model weights um when there is a model improvements between previous so I'm just starting the model training that I have given the training data and the training um ground truth value and then each and every book after it completes the model training we are also feeding in the test the data samples to validate how my model is behaves between the uh the training loss and the validation loss and the so you'll see there will be a four value training loss and the training accuracy validation loss and the validation accuracy so these are I have been running it for a tiny epox but this can be run for uh like 100 a books at the Thousand epochs as well but I told you like we can have a monitoring points based on the early starting conditions so once uh once your model training get completed uh you can also validate uh what will be your accuracy score for your test date as well so uh and and also what you can also do uh I'll show you a one more notebook file um uh you can also save your timed model weight in a HP file which I wanted to show you let me show you that as well so once your model training get completed uh uh you have a a model that say functionality uh you can export you are trying to model it on H by file um so uh when you move your code to a Productions right uh only this HP file alone it's sufficient so you need not to retrain your model again and again so what you can do you can load uh the trained Model H by file directly you can do a testing or be inferencing alone based on the uh the trying to model so not necessary to do a model training array so once you are convinced with your um you are trying validation laws and the training laws and the uh all the metrics then you can save your model in a H5 file or a PB file or HP file and then you can have a referencing code where you can load this HP file and then make the predictions alone not necessarily you need to do a model training on the production environment so I'm pretty much done so maybe I can spend another couple of minutes for the question and answering so any um maybe let me go through the uh the list of questions posted on the chat uh see that is what I told you uh we have an uh early stopping conditions okay so you can set the uh early stuff we have a functionality called as an early stopping condition you can set the patience okay we have a parameter called this in patients um which will uh which we'll see uh how many books I need to monitor whether the loss is not reducing our accuracy is not improving uh based on that you can you can terminate your model training so we don't have any uh statistical way this will be a number of efforts for your business problem so we don't have anything to uh you know prove in automatically to say number of epochs are how many depth of layers and what will be the number of nodes for the hidden layer so that those things will be in hyper parameters tubing so you have to spend your time amount of time on the hyper parameter unit so I've shared my git repository as well let me uh see usually we use a python and like you can use a python based Library like a tensorflow and carers to create a deep learning architectures and you can also use a good amount of libraries opencv and matplotally but to visualize your images and you can also use the sky could learn libraries as well so uh someone has talked about the early stopping let me show you a sample git repository uh which you can go through for the early stopping as well so I think I'll pick that later on so I need to find out that particular Repose so I'll take some time see in video classification so we have we need to use a combination of CNN and RNN as well so you can't use a CNN alone uh if you are going to use a CNN alone for the video classification you have to do an individual frame we also have 3D HDM uh 3D convolutional neural network or you need to use a combination of CNN plus RN and combinations okay the segmentation method include the performance of deep learning algorithm what is mean by segmentation here maybe can you will be deliberate um to perform optimization to the Deep learning new paint losses before predicting the date see the pre-processing what usually we do you can um you can do a normalizations on the images uh you can do like a converting into one zero to one so that that will be the one of the great processes I can also do to try to do a like a super resolution for example if you have a low resolution images try to improvise the uh quality of your images then try to pass on your model so which might also improve your model performance how to do a performance optimization so see there are several factors on model optimizations one will be like you try to use the transfer learning pre-trained models try to see whether your business problem has been already solved with any of the deep learning model like uh you can find out image net pre-trained model Coco data set pre-trained model there are plenty of Open Source pre-trained models are available just try to reuse the pre-trained model weights and then customize only the few layers so that that way you can you know get a better performance in terms of the learning happens maybe a few more questions I'll pick it up segmentation uh you mean a semantic segmentation or instant segmentation so we have an instant segmentation and sigmatic segmentation probably you can refer Signet and the unit architecture so even I have done couple of repository with respect to segmentation you can also go through uh the segmentation as well so I've done it for the second based deep learning architectures for the road scenario so I haven't done it specific to the medical images but I have worked on a semantic segmentation like a unit architectures and the Signet architecture so probably you can go through a couple of deep learning architectures for the same semantic segmentation um object size see it varies so if you are going to use any pre-trained model weights then your image has to be resized uh um resized to the respective Dimension if you are going to create a custom uh deep learning architectures then you can you can Define what will be the image Dimension so if it is going to be a pre-trained model then your data has to be resized to uh the what whatever the architecture which was created basically so is that my screen is not visible people who are attending here from the zoo still that problem persist I don't know so I'll be sharing the video versions of my whatever slides I used maybe you can touch base with the organization see model wise uh so I'll show you one URL so which might also like uh helpful I think we are already [Music] so there are a couple of pre-trained models that are available so um initially as a base model what you can do so uh you can you can go through this website so which has the uh a couple of pre-trained models okay where it talks about vgg resonate Inception um where it talks about what will be the performance or what will be the model size depends on what kind of Hardware you are going to deploy your model so you can you can use any of these pre-trained model note to apply to your custom data set so thank you thanks guys late evening as well