OpenCV Python Tutorial #8 - Face and Eye Detection

[Music] hello everybody and welcome to video 8 in this python open cv tutorial series in this video i'm going to be showing you how to perform a live face and eye detection slash tracking in python using opencv now the code for this is actually very straightforward if you don't really care about how this works because i am going to try to give kind of a brief explanation of the idea behind these classifiers and and how this works so seamlessly if you don't care about how that works then skip forward in the video there should be some time stamps that actually will tell you when we start writing all of the code anyways in front of me here i have the code that just shows our webcam so this will literally just display our webcam on the screen that's all this going to do and i just put this in here beforehand because there's no point in writing it out again i've already shown how to do this in previous videos anyways what we're going to use to do this face detection and classification is something called a har cascade now a har cascade is a pre-trained classifier so something that we're not going to train ourselves that already knows what it's looking for essentially it will look at an image and we'll try to pick out specific features in that image so a feature could be something like the distance between two centroids like you know centroid meaning like an eye or something i could be like the different colors present in a region it could be the the edges or the shapes that are present there's a lot of different features it's very difficult to describe a feature but anything that you could take from an image that is not just the entire image itself could be considered a feature so these classifiers have been trained on hundreds of thousands if not millions of images and what they've done is they've gone through and they've kind of determined which features make up a face and which features make up eyes now each classifier has its own very specific tasks so it's been trained on just faces or just eyes or just birds or just cats or whatever it's classifying and when you feed it an image after it's already been trained what it does is it looks at that image looks for the presence of features that indicate a face or indicate an eye or whatever it's looking for and then if it finds those features it tells you hey there was a face it's present here oh hey there was a bird or an eye or whatever it's present here so that's kind of the idea behind it i will mention that there is some pretty good documentation for this that i'll link to in the description that explains this in more in depth but what i discussed is the very basics it just knows what features make up a specific image or whatever it is it's trying to classify you feed it a new image if it finds those features then it says yep we found it it's located here all right so the classifiers that we're going to use are pre-trained they already exist and we don't have to download them or anything they're actually just by default they come with opencv so i'm going to start by loading these classifiers in so i'm going to say face underscore cascade is equal to and then this is going to be cv2 dot and then cascade classifier and we need to pass the path to this classifier so the path to this classifier is going to be the following it is going to be cv2 dot data dot har cascades plus and then the hard cascade that we want i'm just going to copy this in because it's very long and i don't want to mistype it so let's put it in like that so the first har cascade that we want is our frontal face default.xml so this is the path to where these are stored on our system and then this is the specific classifier that we want that's pre-trained so all you do is just put the one that you want and while it will give it to you so that's what we're going to do for face again this code will be in the description in case you don't want to type all this out and then we need to do one for i so i'm going to say i underscore cascade and i'm just going to change this to her cascade underscore and this is just i like that all right so now we have our two hard cascades loaded in what we need to do now is use them so we can see that we have our image which is being shown here so before i show this image i'm going to convert this image to grayscale and then i'm going to pass that to my face cascade and the face cascade is going to return to me all of the locations of faces so i'm going to say the following cv2 dot cvt color i'm going to pass the frame and i'm going to say this is cv2 dot and then color underscore and then bgr and then two gray like that so we'll say that gray is equal to that line now the reason we need grayscale is because just like many algorithms here in opencv it requires a grayscale image to perform this classification so now that we've done that i'm going to say that faces is equal to and then face cascade dot and this is going to be detect multi-scale and then what we're going to pass here is the image and two other parameters i'm just going to write them in and then i'll describe what they are so this is the function detect multi-scale that you use from the cascade and again what this will do is just return to you the location of all of the faces in terms of positions so what are these arguments well i actually found a really good stack overflow post that i will again link to in the description that describes what these are so before we continue i need to thank the sponsor of this video and this series which is algo expert algo expert is the best platform to use from preparing for your software engineering coding interviews they also have a product called systems expert which is meant to teach you about system design fundamentals and how to ace your system design interviews so with that said check out algo expert and system expert from the link in the description and use the code tech with tim for a discount on both platforms all right so i've got the stack overflow page open i will link this in the description so you guys can read it for yourself but the first parameter that comes after our base image and notice that we're always having the base image first is going to be what we call our scale factor now to go into a little bit more detail about how the har cascade works we're going to be given this we're going to give the hard cascade story an image of an arbitrary size we don't know how large this image is or the hard cascade story doesn't know how large the image is going to be so what that means is that we may have to change the size of this image such that the har cascade has something that it can compare it against you so imagine we give the hard cascade an image that's like ten thousand by ten thousand pixels well it probably wasn't trained on images that were ten thousand by ten thousand pixels so what that means is we need to shrink this base image down and what the scale factor is telling us is how much we should shrink this by at each iteration now the recommended value here is 1.05 and what this would mean is that we shrink the image down by 5 so we'll keep shrinking the image in kind of a scale pyramid fashion that i'm not going to discuss that is linked here there's some resources you can read for yourself but the idea here is that you probably want to put a value about 1.05 the one that we actually picked is 1.3 and the smaller value you pick is going to lead to higher accuracy but slower performing algorithm whereas the larger value you pick is going to lead to less accuracy but a faster performing algorithm so you can kind of mess around with these values and if you notice that your face detection isn't that good then make this value a bit smaller if it's good but it's taking a really long time then bump the value up a little bit now the next value after here is min neighbors now min neighbors is a parameter specifying how many neighbors each candidate rectangle should have to retain it so what this is going to tell us or what the hard cascade is going to do is return to us a bunch of positions of potential faces it's going to say i think this this this this and like you know a million things are faces and they're probably all faces of just like slightly different sizes so this is saying how many candidate rectangles do i need that are say overlapping in a specific area before i determine that this actually is a face so this is pretty much saying how accurate does the algorithm need to be because if it just has one rectangle in an area and saying that's a face well that might not actually be a face that might be a mistake it might have miscalculated so this is saying we need to find at least say five rectangles or ten or whatever number we put here that are in close proximity to each other before we determine that this actually is a face so as it says this parameter will affect the quality of the detected faces higher values result in less detections but with higher quality three to six is a good value for it then we have min size and max size which are really straightforward min size is the minimum size of a rectangle for one of the faces so you can make this you know if you know the size of your face is going to be larger than 100 by 100 then you would just make this like 100 by 100 right and then max size it's the opposite of this this is uh what is the maximum size of a face so you obviously don't want to be detecting faces that are like maybe half the size of your image if you know the faces are going to be really small so you could throw that in but those are optional you don't need to pass those and you can see here we didn't pass a min or max size we just had a value of 1.3 and then min neighbors of 5. so this is going to give us all of our faces so specifically it's going to return to us rectangles that are kind of drawn around our faces or represent what it thinks of faces so what i'm going to do is say 4 and then i'm going to go x y and let me just make sure this is right yeah and then wh in faces now the reason for this is it's giving me a rectangle so i will take the x y width and height of that rectangle and then what i can do is draw a rectangle on my image so i will say cv2 dot and then rectangle and then i will pass my image which we've called frame here i will pass my x y here so let's go x y for this top left hand corner of my rectangle then for the bottom right i will pass my x plus w and my y plus h and then for the color we can pick what we want this to be let's just make it blue so 255 0 0 and then line thickness we'll go with 5. all right so there we go that will draw our rectangle now what i need to do though is i need to figure out the area in my image that is the face so i actually want to grab the face itself i don't want to just draw a rectangle over it the reason for that is i want to use that area so to actually find my eyes because i know that my eyes are going to be on the face so if i find the face then i can pass the face to my eye detector or eye classifier and then it will tell me where the eyes are so i'm going to say that my roi which i believe stands for region of interest and then underscore gray so gr a y is equal to and then this is going to be gray which is my gray scale image right and it's going to be from the following so we're going to start at x and we're going to go to x plus and then w now we're going to start at y and then we're going to go to y plus w and the reason we're doing that is because this will tell us the location of our face right we're just getting this rectangle from our grayscale image and storing it in an roi grid now we're going to do the same thing for color so roi underscore color is equal to and then frame and then again same thing x colon x plus w and then comma y colon y plus w or y plus h what am i doing here okay so now we're going to get this area from the color image which is the frame and you'll see why we need both of them but we're going to pass roi gray to our new eye classifier and then we're going to just draw on roi color the um what do you call it the rectangle and actually i just realized that these need to go in the other order i always make this mistake with the slices we need to have y go first because of the way that we're we index values in this array so let's do that and then think there we go okay awesome so that should be correct we need to do y first because it goes rows and then columns common mistake now what we do is we're going to look in this grayscale image roi gray for any of the eyes so i'm going to say the following eyes is equal to and then i cascade and then dot what is this what do we call this detect multi-scale then we're going to pass our roi underscore gray like that then we'll pass the same thing 1.3 and 5. all right so now that we have our eyes we want to draw all of these eyes we're going to say 4 and then i in eyes like that and actually let's change this so that we're going to say e x e y e w e h so i x i y i i width i height then after this we'll just draw a rectangle so let's say cv2 dot rectangle we'll draw it on not frame but roi underscore color i'll explain this in one second then we're going to draw this at first e x e y and then bottom right hand corner is going to be e x plus e w and then e y plus and then e h then our color will make this green so 0 255 0 and then line thickness can again be 5. all right so now that we have this this is actually all we need to do i'll show you this working in one second let's do a quick recap so we are going through and we're going to just look at the frame from our webcam right we're going to convert that frame into grayscale we're then going to find all of the faces in that frame we're going to loop through all of the faces which are really going to be rectangles and then we're going to draw a rectangle for whatever this rectangle represents we're just going to draw the face essentially our box around the face then we're going to figure out the area in the image so the actual pixels in the image that represent our face so we get the gray and we get the the pixels in the colored image now this right here is actually a reference to the original frame so if i modify this here this will actually modify the original frame so keep that in mind so the reason i'm doing this is because i want to find the eyes only on the face right so now that i have the face we find the eyes on the face but this is going to tell me the location of the eyes well at least this will on our roi gray now roi gray is not the same size and it's not the same image as our original frame so when i draw my eyes i draw them on my roi color which is the same size as my kind of grayscale portion right here so what that means is that i will actually draw them in the correct location now i know i'm explaining this kind of horribly but the idea is that if we were to just draw this rectangle right here on our actual image if we were to draw it on frame so all our eyes we drew it on frame and we didn't draw it on roi color these eyes would be in the wrong spot the reason they would be in the wrong spot is because this is telling us the location of the eyes on our roi gray image so on our face like you're just looking at the face it's telling us the location of the eyes relative to where the face starts so we need to draw it relative to where the face starts so on roi color not on frame otherwise they're going to be in the wrong spot so hopefully that's clear but that's why we're drawing it on roi color because roi color is the same size as the image that we detected the eyes from then what we do is we draw not the image this sorry should say frame like that and then what we can do is well just run this so let me reactivate or deactivate sorry my webcam and let's run this and let's see if this is working all right so there we go we can see this is indeed working i can kind of move around a bit you can see it's tracking my face if i turn my face notice it doesn't really keep track of it because it doesn't obviously detect the side of the face and then i can open my eyes i can move around and well you get the idea this is the live face and eye detection in opencv and in python so i think i'm going to leave the video here that's all i need to show you in the next video if there is a next video i'll see if i want to do one or not we may potentially train our own har cascade classifier if you want to see how to do that leave a like on this video subscribe to the channel and also leave a comment and let me know and i may potentially make a 9th or maybe even 10th video in this channel where we do something like that so with that said if you guys enjoyed make sure to leave a like subscribe to the channel i will see you in another youtube video