LIVE CODING: Face Detection with Python

Key Takeaways

well hello there it is Sunday November 27th 2022 I hope you're doing well I hope everyone out there had a great Thanksgiving I definitely did ate a lot of Turkey we actually had a not so fun event occur on the actual Thanksgiving but luckily our family celebrated pre-thanksgiving with some extended family so we got it in um unexpected was uh we had I didn't get coveted but my wife got coveted so I was going through my family I haven't gotten sick yet I've been kind of lucky but uh she's been uh regaining energy and I'm ready to code tonight how are you guys do we have some people in the chat if you're not there yet uh if you're if you're just watching the recording feel free to skip forward if you are in the chat on on um YouTube feel free to jump over to Twitch I'm gonna go ahead and throw my twitch Link in here that's the best place to Strat to chat Strat I don't know I said Strat um I did change my YouTube name just to my name I change it from Medallion stallion or wait Medallion data science to my name I I at first started creating stuff on YouTube and I was a little self well I wasn't just a little I was very self-conscious of of actually making stuff for YouTube because I thought it was a little strange and then um you know so I I picked this this random name that really wasn't associated with me but now I feel like that barrier has been broken and I've kind of gotten through the hump of being worried about that so that's why I switched it just to be my name I might change it to something else later what do you guys think do you have any suggestions I'm doing well thanks for asking krisha um uh no sickness on my end I don't know I might have just passed over me but uh been very close quarters with my wife and and I would think that if she tested positive and has been feeling sick then I've been exposed so I don't know we'll see what happens we'll give it some time time will tell um how I hope you all had a great Thanksgiving and I have some ideas for what we're gonna do hey we have someone named risu and it's a big fan of my work thanks I'm a big fan of you for being a fan of my work um let's go ahead and start up a poll so I have for tonight three ideas that I would like to maybe look into um so I'm going to talk about them first so the first one is I would I just made a video about on my YouTube page which by the way if you haven't checked out before exclamation point YouTube and twitch will bring you up the link or it's YouTube .com Rob mola we'll take you to my YouTube account and I made something about stable diffusion um 2.0 so we could look into that and I want to see what the deal is with these custom trained models and and what their how they're trained and if we could train some I attempted to do it myself but um before and it didn't work out too well so I can show you the results of that so that's number one the first option that we might work on tonight option number two is uh I'm just typing it here so the number two is um I have plans to make a YouTube video where I just quickly summarize uh two different videos where I just summarize key Point face key Point detection algorithms for python or that at least have Python apis and there's like two main ones and then also pose estimation libraries that are out there there's a few main ones for that too like uh Alpha pose and open pose being the main ones and I started working on some of that on stream before but I forget where it went we never finished it so that could be option number two to look into those libraries and to kind of make a comprehensive idea of what's out there and then I also have my second webcam set up here and we can test out and see if I could get them running on my machine and maybe you guys can give me some fun ideas uh option number three is kind of out there but um I was thinking about this earlier in the week and now I'm a little bit less hyped about it but it would be to work on with a new programming language to actually move away from Python and to check out the go Lang uh uh programming language to see if I could do some data science with it because I know that there's some data science libraries made with go and I want to see what the differences are I was thinking initially to do it with the rust but I don't think rust is like made for that sort of stuff so I'm going to start this poll on Twitch and I'm going to switch over here I'm going to switch my display over here to show the results so a lot of people already voting for key points face detection that could be cool let's see what people are saying in chat uh I think the changer name is a good idea oh thanks Gustavo I appreciate that hey good evening what are we gonna do today good question you probably asked that while I was talking about it so if you check out on Twitch you can um vote on what we're going to work on to what how to be top one percent in any kaggle competition reesu who asked that question the answer to that one is just spend a lot of time um found you from YouTube learning data science myself excited to join you thanks for being here live that's awesome what is your OS and desktop environment that is a great question out there ain't my problem z um I can actually I did exclamation point YouTube to show you guys and I'll also add it here www.youtube.com backslash at Rob Mulla this will bring you to my page hey what is that sound that sound means that someone subscribed on on Twitch thank you so much for subscribing we haven't even done anything yet and you're already being so nice to me thank you so much hey we got two so that's something I was gonna talk about here uh soon before we got into it but hey you guys beat me to it um I switched up the wheels so the wheel is a little thing um so thank you for subscribing with prime Ramen Ramen and clipped subscribed for eight months clipped has been with us thanks so much for keeping up your subscription so I spin this wheel every time we get a subscriber I've been thinking about I I kind of changed up what's up on on here uh because I want to keep the Stream flow moving a little bit more but I also want to make people not like scared to subscribe because I definitely would love that okay so type Pizza 10 times we can do this quick uh look I've already done it two thousand times so I've reduced the numbers of things that I have to do and I also kind of lowered the likelihood of having to do things like push-ups and uh added some more so if you guys can think of some good ideas for what I can add to the wheel for when people subscribe let me know I had a dad joke uh signing into the mic screaming Kevin is still a good one and look do nothing it landed on so that was for you clipped I'm doing nothing especially for you uh let's look at the survey we still got time to go please put in your vote here it looks like key Point detection is winning at this point eat a whole pizza I had to eat a whole uh I had uh eat a whole banana at first and the wheel just just for a little bit of uh my streaming history the wheel was something that I initially added to try to get followers so anytime someone followed I would um spin this wheel and do 10 push-ups so I would have like a bunch of people following uh per stream as it started to get more popular and then I changed it to just being subscribers so uh that's how far we have come that's how far we have come um again I already gave a shout out to our um YouTube but while we're waiting on the results of this survey I also have a Discord which I'm going to chat out here um and just a reminder if you're in the YouTube chat please join us in the twitch chat if you can that's the best way to like keep everything together and do I use go a lot I've never used go so the question would be like try to figure out how to even install go um but I use go rust more frequently oh wait Anthony is saying came from YouTube since you didn't see my chat rust and polar's crate which is basically pandas oh yeah yeah so polars but has a python API right so we could do Rust stuff too we could look into rust I do like the key Point detection pose thing because then it kind of works towards my goal of making a YouTube video about that too um it would be really fun to watch my Learning Journey learning go uh I think you yeah I mean maybe you might be overestimating how fun it is to watch someone struggle but we we could see I mean I'm gonna do whatever the survey says so you guys choose which material do you recommend to study computer vision so Ramen that was a great question I started working with computer vision honestly through doing kaggle competitions so um I would recommend checking out some of the old cargo competitions and reading some of the winning Solutions um some of the guys that I've teamed up on in the past on kaggle are just some of the best guys out there for creating so I guess it depends what you want to do with the computer vision because computer vision is such a broad field um but let me just jump here to show you guys so I've dropped in my competitions ranking man I was at 31 and I just haven't been able to spend the time on competitions but let's go to the rankings uh Dieter is first his real name is Kristoff he's first overall um and if you can find any of the code that man where is this okay there and PSY and all these people are just really good at computer vision or just data science in general so um they published their winning solution to which competition was that I think it was this Google Google landmark solution yeah so obviously you need to um you need to just like get your basics down of being able to read in images you can watch my YouTube video on how to work with video data and with image data on YouTube I have those here so like uh self-promotion stuff it's here uh python let me just Google python image data processing hopefully it'll come up here yeah it's here at the top so this check out this video which is on just how to work with images you need to have a good understanding of how to do that but then if you want to learn about architectures um oh this sungli's like these are some of the interesting models that are out there um for how to do that are like the state of the art for training models and then you kind of learn about things once you start getting into deep learning if you're doing like classification you learn about augmentations the augmentations are huge um we're in training models um you learn just about the main architectures that are popular like uh efficient net or uh I don't know some of the new like vision um Transformer based models you basically just test them all out and learn how to like train the model with a good learning rate scheduler and what else so that's just like image classification and then if you want to do like key Point detection stuff I don't really know I mean a lot of that we just inherit other people's models uh hey Mr Gabriel welcome to the chat hope you're doing well how to learn feature engineering any course recommendations I don't know of any courses that specifically talk about feature engineering I'm I should make a video about that but generally you you just need to have some idea of how the data works in order to make good feature engineering so it really heavily depends on your data set I mean you can always do aggregations and like Macs and men's but um I think that an under appreciated part of feature engineering is making features that are actually helpful to non-technical people so let's take the example of like if you're creating features for a model that helps Runners know like uh what contributed to them being fast in their run I don't know like a model that predicts how well they've uh how many calories they burned you want to make features that are easily explainable so that when they see the results of the model and you tell them the future importances and stuff like that that actually um helps them out now that's a not a great example but you I think you get an idea of what I'm going at with is go growing for data sent data science I don't think so um but I but there was a short period of time where the people said it might be the Python Killer uh but okay so we we landed on face and key Point detection we're not going to talk about stable diffusion um and data science would go maybe next time so let's are you planning to do Advent code see the thing about Advent or code that's a great question I've never participated I've seen other people stream Advent of code it does seem like it takes a lot of time and typically around that time of year kaggle releases an optimization problem around Santa and I like to participate in the kaggle competition because I find those more I don't know I like I've never tried the Advent code but it seems more coding based like just and and less of like optimization or machine learning which I enjoy more so I wish I could do both at once um I saw a few days ago that Russ will dominate the data science field see I was trying to see like not to get sidetracked but I think Russ is more like data engineering and maybe that's just because I came when I Googled it I came across this um looks like there's even a video about it but I don't know if it has the packages needed to really do machine learning but maybe I'm just totally ignorant to it um I was figuring if we were going to do the go one that I found a few different um tutorials that would walk through and basically walk through oh look here's a rust tutorial so maybe I should have changed that for rust yeah it looks like rust might have been a better idea than than go so maybe next time I'll talk about rust because you can even do these plots look at that admin code doesn't take too long some of the later days do need some optimization ah maybe it would be more fun to do Advent code I might give it a try when it starts off but like I have this tradition of working on usually on Christmas around Christmas time I work on these like where is it Santa here Santa's Workshop tour uh last year's one was really fun for the Santa one it was like an optimization one where you're trying to get all these uh emojis in the right order but yeah these these have always been fun I haven't always placed very well in them but um but these optimization problems were are fun around Christmas time too many fun things to work on okay so let's go ahead and get started here now I do have I made sure I was running stable diffusion uh with the pi torch light being Library there in that winning window that's what I just killed but we're gonna make this a little bit larger and make sure that everything is appearing in our screen that's important we don't want to hide stuff from you all there we go is it like Google hash code what's Google hash code would you recommend the kaggle learn series for newbies for sure yeah I think they they're good at what they're doing what they're showing and also Abhishek has some videos like to get started with kaggle which could be helpful um check out his channel I think it's linked in my YouTube channel uh I think they're somehow related to kaggle learn I'm not quite sure all right so let's go here let's go into my repos directory ah twitch stream stuff and then make a directory for o55 um pose key point and pose all right let's make this a little smaller go up a directory and do Jupiter lab here do I don't have Jupiter lab already running just making sure here no I do not all right and now I have my Jupiter lab session here we're going to go into twitch stream projects here we're starting with a a completely blank slate here um which should we start with chat what should we start with um face key Point detection or should we just go straight to pose see what you guys are saying in chat face okay face so I've already done this on stream before at least in a YouTube short before uh let's see what people are saying Google hashcode is a team event about solving an optimization problem it sounds similar it would be similar so like when you was traveling salesman actually the last years turned out to be more like a traveling salesman problem Russ might replace a low-level knee for C plus situation if you need the C plus plus speed you don't see it replacing python gotcha but python is built on C right I'm currently work in retirement savings analysis any recommendation with kaggle data related to this my only recommendation is just to search the kaggle website we might see some stuff have you won any kaggle contest so the best that I've been on a team is second place in an NFL competition and uh yeah that's as good as I've placed but as far as I'm concerned when it comes to kaggle if you can place in the gold medal range that's like a win that's very hard to do and it takes a lot of time and determination so yeah never one all right so phase key Point detection I do have this second camera set up here that we could use for my for um doing the key Point detection so I need to figure out uh my devices but we'll do that next so uh python face key Point detection I want to save media pipe for the end um but before media pipe I know there is d-lib so let's look up d-lib face recognition python so this will just detect the faces oh geez um this isn't dlib though this this is fit the face recognition Library there's also a face yeah we can do face recognition so let's check this out so uh conda activate kaggle two Because deep face is another library that I've used before let's yeah let's conda deactivate this and conda activate deep face and maybe since I already have that set up we can test that out first Louis thank you so much uh using opencv I think some of this stuff uses opencv backend opencv has uh definitely has um a way of detecting faces but I don't think it does the key Point stuff so the way I understand it is deep faces a library for python that actually is built on top of a bunch of other models so it can do stuff like verifying if one face is the same of the other it can obviously detect wear faces in an image but here yeah so it's using these back-end models which you can choose from and there's a lot of different options you can choose from and we'll go ahead and try out a few of these then it also does some stuff that I've noticed it does it it's not that great of a result but it can it can just check things like your race gender and emotion the emotions really subjective so I at one point this is when the Johnny Depp trial was really really big news everyone was talking about it I tried to take Amber Heard and Johnny Depp's video footage and run this detection algorithm on it to see if there's interesting stuff to see if that would be a cool YouTube video I recorded the video and then I didn't think it was good enough to release um but yeah so this has media pipe built in it looks like now it also uses dlib so it's kind of like everything in one place but I don't know if it's necessarily you know the best Library we'll also go in there and just use media pipe itself so first thing we need to do obviously is import deep phase actually before we do this no module named tqdm pip install let's upgrade deep face pip install tqdm it says it's already required so what's going on here okay there it just it just had a hiccup um we're also going to import CV2 obviously so CV2 is like what we can use to pull in from that camera um so let's create a capture device I think it's I don't know it's Auto completing um and then we need to figure out what which oh video capture is what it's called and then we need to find the device number so I always forget how to do this but there's a way yeah so of course that I have already been to this site and there's a way on Ubuntu to list all of my devices in command line so this will list all of my devices so I do have I don't know what this camera is um I do have this old UVC camera the Brio which I'm using to cast on and this c992 which I believe is this other camera and we could see this is a video device seven so if uh it says it here do you guys see that so we're gonna go here uh give good evening casual human welcome to the chat if you see that over here we're going to set this device to be seven and that's going to be cap no cap on this and then we can do cap.read and this is going to give us our image and our result I think so no no so it's going to be result and image IMG now if we do image shape yeah we have a image we're also going to import here import matplotlib and we're going to show this image just to make sure it makes sense I think I can do this let's see if this works instead of converting the colors okay so the colors are going to be messed up until I convert them here yep there I am look but my colors are weird that's because I need to convert colors from BGR 2 2 RGB and then I should be able to do this okay that's me um let's also make a size of this I'm doing this all from scratch even though I've done a bunch of times I think if you've seen my other streams you might have seen this before but let's make a 10 by 10 and then let's also ax axis off there we go that's the picture of me um of course it is needing to be converted again so that's Picture Me Do you guys get it so we we got the camera set up ready to go um might have seen the past before you got a cam dry image uh yeah I guess that's another way of doing it bro is a Smurf because BGR not RGB yeah so CV2 uses BGR uh matplotlib uses RGB so now I'm looking at the camera see and I'm reloading the cell I'm going to try to thumbs up and do control run this so you can see I'm not faking this that is my real second camera and you can see me in it um there we go so we have that image capture device set up ready to go and now we got to use deep face on this we're gonna do some emotion detection and some other stuff let's see if I can make it show how happy and sad I am okay so we have all these different back ends we have to use from I believe some of them are just for detecting the face so we're gonna just do this one by one all these options for back ends can I load always like to load this extension to make this easier to read okay so yeah this is not gonna work just off the bat right off the bat so let's go ahead um that's another thing so face verification this is the verify that two images are of the Fate same person and it it requires it to read an image uh someone's saying they're not able to implement after they're learning absolutely yoked arms I don't know about that okay and I don't even know what that means so what am I gonna do here do I have to save this to disk which is kind of annoying see if I just give this like image and image it's not gonna work because it wants it to be a path and it's trying to go in there and read that path so it's going to try to take this image as a string and then look up that path which will not work oh wait it did work so I guess they have the implementation able uh okay so I guess even though this is image One path and image two path it can take in two images um yeah so let's do this let's do this for uh X in range five this Loop and we're going to import time oops we're not going to plot it yeah we're not going to plot it we're going to make a function called plot CV2 image and I've done this million times before and that's going to contain all this stuff which we will then use later on if we want to we're going to create an images list and we're just going to append the image and then we're going to time sleep for one second so this is like this is like when you go into the photo booth and you're just letting it snap a few pictures of yourself and it's going to store them out off uh I'm sure all of you are too young to know what I'm talking about but trust me back in the day there were photo booths that you go in to get pictures taken of yourself um all right oh wait I guess it's taking them is taking them I should make different faces can I cover some of my face I don't know okay so now how many images do we have uh they all say none oh so I think I needed to release this capture device otherwise I was trying to create the same capture device so let's do print smile and then Flash all right I get a look at it I don't know if it's working or not yeah that's not working um I think it's having problems with that so let's reset this whole thing see if it gets any better so many thumbnail phase all right so cap release does not work here clearly so I should I think I should just Define the capture device once so it doesn't like that all right all right so images there are five of them now four image in images plot CV2 images what they're all the same I barely moved I they're all of me making this weird face did I do something wrong here maybe I need to make this sleep like five seconds five seconds might be way too long I'm trying to like make it so you can't see my whole face hey we got a subscriber on Twitch yes yes yes who is it who is it thank you super LOL I thank you let's spin the wheel for you hope you're enjoying this scream Kevin as loud as I can this is one of my favorite ones Kevin all right just yelled it it's especially applicable at this time of year right because of Home Alone is there something wrong okay so that's different ah they are definitely different now I look crazy in these I look insane but maybe that's just the way I always look and I don't realize it I needed this stay still more the Kevin stuff don't scare your kids I I've never heard them mention it before so I guess I'm far enough away that they don't hear it uh definitely thumbnail pics casual human knows what's up all right now let's do this let's see if image so before we just showed it the same image twice and we asked it to verify if they were the same face but now we're going to show these two different images so before like the distance between the two it said it was basically zero like this number is almost zero um face cannot be detected in the second one oh so I was being a little bit too crazy with this stuff I found a face there but not there all right let's try to retake these photos it's also add a fig size here let's make it five by five make it a little smaller all right I'm gonna try to stay still [Music] [Music] I can also just pull up a thumbnail or something I shouldn't add the sleep after Okay so let's try this again did I do this again why did I cover my face yeah of course it doesn't see an image in the second one so having a hard time finding a face in my second or in my first one yeah because I wasn't looking at the camera what an idiot all right let's try this again all right there we go oh there we go I think it's done this time something's weird going on because I did not put my hands up this time I reset images then I append images and I'm reading maybe I need to like flush the reading in maybe it's a camera buffer thing let's try this it is a camera buffer thing because it like took five before these are the old ones I think it's above buffer thing what if I try this this should just pull 10 frames and throw them away away before it takes the Flash and then we'll make it sleep right after this but not for so long like two seconds is that yeah two seconds all right so it's throwing away some of the buffered stuff Okay so I think that's working better the last one is new but these ones are new too I think that everything from that so let's try this again all right so we got a bunch of different ones all right now they look somewhat different let's see if it can detect faces what the crap oh I didn't append this list this time why did I take this out let's do it again all right all faces all kind of different all the first two aren't different so let's see the first two and then we'll print what the result is so it says these are very close verified as being the same person oops back end that was using back end retina face let's do two so now the distance is a little bit different yeah I see the distance is higher but it's still seeing that all of them are the same face and actually like the difference between this last one and this first one pretty much these first two are like identical right um so I'm surprised that zero index zero and one didn't have like oh yeah I guess it's like a .02 so it's it's saying it's a very close uh let's find an image let's find an image of Brad Pitt and let's get this image and let's see if I look like how much it says I look like Brad Pitt 20th century Premier Red Square my goodness so look at this the threshold here is 0.4 and it's I'm saying that my distance from Brad Pitt is .41 ah I'm so close to being Brad Pitt so that gives you a little bit of an idea of how much to trust this like I'm not gonna put my bank account information um oh look at this one oh shoot I'm not going to use like this to protect protect a vault with my bank information that's open source for you two it says that I am the same so what is the two images two so it's saying this looks like it's saying this guy and this guy are the same verified AI says I'm Brad Pitt um what if we try one of these different back ends so we have we're using back end four which is Retina face I'm not sure if all of them have like um a verify okay so it looks like opencv at least has me being uh further off yeah not even close so I trust opencv at least when it comes to my face versus Brad Pitt and then image two path let's make this images zero so this is me against myself let's see how it does in rating that 0.17 thresholds 0.4 so this is uh this is a little bit better does it look at the distance of eyes mouth and nose I don't know I think it so let's go into the deep face it's a good question let's just go into deep faces repo um so these are some wrappers for all the models dockerizing it let's see if they say this function verifies face pairs as the same person or different person it expects that exact image path and inputs foreign passing numpy or base64 encoded image is also welcome so we found that out but I wasn't sure at first then is going to return a dictionary you should just check the verified key it doesn't really tell us any more than just that base models foreign so it's like finding the euclidean distance of two two faces but I don't know exactly what it's feeding in Source representation and test representation oh so it's it's finding the cosine distance of whatever this representation is it could be key points I'm not quite sure I think media pipe is pretty good though so let's try to do media pipe as a back end so negative one and see what we get all right it doesn't work with media pipe no module name media pipe so it must not be an available back end for so let's remove this to my image two so that's opencv oh SSD is actually downloading weights it says I'm not Brad Pitt but I'm close 0.56 I mean I guess I could look very very different than Brad Pitt mtcnn so that's this model so the mtcnn does say I'm the same thing as per pit and so does retina face and then media pipe does not have it uh a metric verify on base cosine similar someone's saying in YouTube hey Rob checking in from my job hey I hope you're having a great time at your job all right so this is the most basic stuff now let's just see Finding faces this seems like okay so the way this face find works we would need a database we would need a database that existed with with a bunch of different faces in them and then it would look up and be I think do the cosine distance between all the faces and try to verify which person you are so I don't want to go ahead and create a whole database foreign 50 chance of Brad Pitt yeah just the one picture though like all of them are not convinced on Brad Pitt but it is this one with a real big frown that really gets it confused maybe because of his his uh beard there yeah that's a tough thing like if you're trying to elude the face detector there's a lot you could do I would think to just totally screw it up like for instance for instance uh grow a beard or something like that all right so now let's do some facial analysis so I believe this is the sort of stuff where it's going to tell me my uh gender and age and all that stuff all right so code editor outer sync okay so it's detecting my emotion my age gender race and then race again I guess it was really confusing I mean I am a mixed breed so all right this is what it's finding for this image this first image where I'm smiling a very big smile like I'm trying here guys it says I'm angry oh wait no no no no no no no no let me take this emotion it actually tells us the the rating so I'm going to import pandas here let's go up to the top and then I'm going to go way down to the end make this into a data frame our index is going to be zero uh I think it's Orient index maybe something like that what is it uh I know this works index equals zero and then we'll just put a t on this to transform it and rename columns I know this is a lot to just try to get this prediction so this is our emo DF emo data frame see how emo I am music is weird um okay so all right this is legit it thinks that I'm mostly happy there's also sort values uh by prediction lowercase's prediction all right so mostly happy and neutrals next but mostly happy this makes sense right I don't know why I made this plot so big not emo yeah did they have Emo as one of the I could try to make this side by side too so let's make it let's make a function geez just close this where it takes the motion data frame and let's make this axis Now 1 by 2 what is this gonna do and then the second one emo DF will do this plot on the second plot I don't know if this is going to work let's also make this 10x5 all right something went horribly wrong oh I did Plot show too early there we go there it is that's my uh my why is yours why is my screen got wider I don't know 15. oh why is this fig size not here directly all right now we see my image and the emo data frame so how can we do this in a loop we actually need to do this don't want to plot it down here and we'll do the the demography is that right on each image so I think I made a few different emotions there let's see if it can pick it up oh also some of these have like a way to disable this progress bar let's make this progress bar faults I've noticed it still shows some progress bars but not all of them all happy this okay sad it does think I'm sad do you have any experience in trying to learn the API which there's no documentation that exists basically just scripts and some more or less anecdotal Docs can you do you have access to the source code oh why did my screen get wider I wasn't paying attention to you yeah that sorry sorry guys you were right I should have paid more attention to you I took you all for granted but something went wrong on that screen hopefully it's fixed now yeah that's tough one against I don't know quite exactly how you would do that all right it thinks I'm happy in these first two which I think are identical anyways thinks I'm sad here and kind of angry surprise look it got it I'm in Surprise is that really surprise let's take a few more all right I'm going to try to do each of these happy neutral surprise sad angry fear disgust let's try to retake some pics I need to take some more I think what I need is a longer sleep and let's take like neutral heavy sad is it done yet is it done yet I think it's done the first two are always the same because I'm it takes me a while to figure out what's going on um and now let's run it on these so I'm trying to make some of these so this was trying to be neutral but it thinks it's sad no I I mean I don't know really the difference between these two images but for some reason I'm sat in this one in neutral and that happy sad surprise I didn't do fear sad again angry essay this is I don't know surprise I suppose I mean I think we've showed what's capable with that right um so that's just that's just emotion emotion recognition I did have to search up the spelling to make sure I didn't embarrass myself there I think you should put the sleep after the print you're probably right let's see what I was doing here I was so it's like takes it then it sleeps and immediately then it says flat Smiles Flash and it should read the image from then oh I think that this sleep should be up here I think you're right about that pause then clear the clear all the old frames and then smile take it yeah I don't know we have enough pictures of me with the motions um so we did facial analysis oh yeah but we want to look at some of the other demography stuff um so it does pull out the dominant emotion just as a value so that's just the arc Max of this so that's not that interesting region this is just where it detects my face let's see what age I think I think I am well it thinks I'm 32. I've noticed that this age number just jumps way around I can only see Brad Pitt with different emotions you're hilarious you are hilarious um Race White mixed race but I'll let it slide should have uh it should have 0.5 Indian 0.5 White gender Man Okay so let's see let's see if the age actually jumps around as much as I thought it does actually let's instead of mine I mean we could run it we could run it on this just to make sure that it's around the same is the age right no it's off by like eight years seven years okay so here we go and we could pull up some celebrities Maybe so these are things that shouldn't dominant race I just like it's such a weird weird way to say it uh uh gender all right 33 white man this is 32 38 39 hey this one got me this one got it so it's in the ballpark and it's not saying like I'm some crazy things um yeah you're right back ends do matter here how should we do this so let's just do this print that is a great point that is a great point let's just do this thing is I don't know if all the back ends are gonna have the same output format Okay so opencv face cannot be detected using opencv try except is this a specific exception except value error I'm not quite sure it's actually using these correct back ends no module named D lib oh do I need to install d-lib let's try to pip install d-lib well how about that I didn't even have d-lib installed wait I'm 39 yep yep for for about six more months is this an interrogation of my age I look younger yeah let's hope let's hope it keeps that way it's fine being old means that you're wiser right for now let's just do this mtcnn's going crazy look at all these different steps it takes hey this one thought I was Middle Eastern that's that's closer that's got some of my retina face and media pipe broke probably because I don't have it installed I'm gonna have to do that next all right so we got all that running face detection and Alignment this is boring so I don't want to do that dues hey Rob you taught me so much about Panda's I watch all your videos enjoy teaching style keep up the great work thank you so much Deuce I'm so happy to have you in the live stream that's awesome I'm glad that you found some gain some knowledge from what we've been doing tonight's kind of more goofy but all right so what do you guys think in chat should we focus more should should I run this um should we compare the detectors like the motion age and all that stuff detectors on some other images like we could pull some images of celebrities or should we continue on and see if I can get the key Point detection on a live camera working so we can detect uh facial key points let me know in chat if you have a preference otherwise I think we'll move on because I don't know where I'm going to download all these images of celebrities from although I knew I know I think I created it something on a kaggle data set but whatever all right no preferences anyone in YouTube saying anything nope has anyone even there do what I want okay cool let's do key Point detection using d-lib so um facial landmarks with d lib and opencv so d-lib can find in two dimensions a bunch of different key points eyes eyebrows nose mouth and jawline so we did import dlib or or install d-lib let's see if I can now import it yep that worked um let's copy this image and see if I can paste this in here so these are the different point oh why is it showing it twice these are the different key points that it would detect in the face so it's a lot less than media pipe which we'll see in here in a second but I I don't think I've actually ever used dlib so they're just doing argument parse errors okay so this is how they're doing it they're creating a detector which is get frontal face detector let's see what get face chip get face chip details get frontal face detector get histogram and get wrecked you guys want to get wrecked no let's get the face detector and the predictor will be shape predictor what's this shape predictor that's an argument it wants passed through path path to facial Landmark predictor all right so I have to download this this looks like a little bit better of a okay so IM tools okay this person's using the video capture device and they're showing their so this is just what we've already done all right so they downloaded this shape predictor face landmarks.dat and it's saying to download it here download that and we're gonna go into this directory CD into this directory and then we're gonna move from my downloads this shape predictor into this folder so now it's here right next to everything else and I think what we'll do here for shape predictors all we have to give it is this local file which is called shape predictor landmarks there we go all right so we've loaded in created detector and a predictor Adrian Rose Brooke is that the person who wrote this should I know this person I like the pi image search content fortunately the paid stuff is costly for Brazil oh no okay all right so what are we going to do here shape predictor is p which we've loaded in oh so these face utils look like they might be helpful for plot for actually showing this d-lib results so let's do pip install I am utils and let's use this guy's example from imutil's import this okay so we can just do this all this stuff except for image is equal to images zero it's going to convert the color to grayscale it's going to do the detection on the grayscale and then this should just hmm it's writing the circle on this image oh and then we need to do plot CV2 image on this image there we go do you see that hello how's your day going my day's going well how was yours hey elusive Peach it's been so long elusive Peach is back in chat good to see you hope you're doing well tonight we're just doing some facial key Point recognition Rao hope you're doing well too all right so for each of these images looks like we're seeing things fine see how it huh why is it having a little harder time on this first one um and then so basically let's do it for all of them yeah so here's all the different ones different Expressions it's pulling in and I mean it's doing a pretty good job of finding even through my glasses um we can make this size a little bit bigger to see number 10. I'm not quite sure why it's doing kind of doubles there but these numbers should align up with these key points right so if we look at Key point one it looks like yeah so the length of this shape output which is the length of the predictor is 68. there are 68 points here let's say if the if the if we're at Point if I is greater than or equal to 51 it might be 50 because of indexing color is going to be let's do red oh yeah the color is going to be zero um RGB BGR so I think it's gonna be zero zero two fifty five else color is going to be equal to zero two fifty five zero and then we'll put the color in here um so hopefully you guys get what's going on in here we're just looping through each of these key points and drawing them as circles on this image and and um I want to take all of the lips key points and color them differently so the way I'm doing that is I'm just assuming the index is going to show us the number oh and it's not working oh because that's just going by the rectangles what is this so I think what it's doing is it's finding two faces this 4i needs to be in yeah so I need to do in here it's going to be four key point x y and enumerate shape there we go it's like reddish green for some of them why are there multiple ones length of shape is 68 I'm just confused about why there it's also adding these green ones behind the red images it's almost like it's detecting multiple faces and then making some of them green and some of them red yeah it does look like greater than you're right Mr Gabriel it's like more than 59 or maybe more than 58. but I'm a little weirded out by why it's doing that oh do you know what it's do you guys know what's happening I just realized what's happening do you want me to explain what's happening anyone care okay so what's happening is um so this code uses CV2 to draw the green dots on the image and every time I'm running this cell I'm drawing more dots on it I'm never they're never going to be removed and it's probably having a harder time detecting the face because of all these weird green dots so I should have done what I should have done is this dot copy I think that would have fixed it but now I've ruined all my images they're basically gone so in order to try this again I'm gonna have to take a few more pictures unfortunately does that make sense yeah I need to take new picks in order for this to work again so let's just take Let's Take Five hey mgx thank you for subscribing again that's awesome of you I appreciate it it's been six months of you subscribing on Prime that's so awesome of you by spinning the wheel for you and I'm going to take a break from my beer and drink some water for you there we go um so now if you if you haven't been here before mgx what we're doing is we're detecting key points on faces there we go it's doing a little bit better now it's a little bit off on my lips there and not quite getting the jawline I think it's a little confused is something went on in this first one just making it too weird of a face but it's pretty good right two dimensions 68 points and I'm impressed now let's do some media pipe why am I making this for fun I'm just I'm planning on making a YouTube video gerbo where I explain the different uh facial recognition software that's out there just like as a tutorial so I'm using this stream as a chance to make myself familiar with what's out there so we've already run some of these models that can do like emotion detection trying to detect if people are the same as each other too which it failed miserably on because it thought that I was same thing as Brad Pitt um and then these emotion detections actually did pretty well and also the gender in uh race and age stuff for me it worked well but that doesn't mean it generalized well for everyone I know that there's like systematic systematic biases in a lot of these models that we have to be aware of when we're using them uh like people with much darker skin tones if the model's not trained on those then there can be issues there's also issues with cameras just in that in that um that same vein so but at least when we're testing off on me uh yeah we're checking out so I'm curious what tools do you use most at your job that's a great question gerbo check out my YouTube I actually explain my whole setup in this video that I created and released not to recently ago because I got the I've gotten this question a good bit so this ultimate coding setup guide is a video that I recommend you check out um I talk about like what I use uh just pictures of Brad Pitt popping up left and right if I can find this chat to put in right there um yeah so uh but I do use opencv a good bit in my work media pipe let's try to do media pipe and then we'll call it a night all right so I think I already have some I already have a media pipe maybe not there let's see in my GitHub repo all right so I have this video capture device going and we're gonna import some stuff that we already have here so we're going to import media pipe as MP oop medium pipe is not installed so we're gonna exit out of here we're gonna pip install media pipe into this conda environment so I make sure I can access it there we go and then let's see if I can import it now yes I can let's get a different music Vibe going on here not that one all right so let's uh let's try out media piping let's see how it works so in order to use media pipe you need this these solution package which has the drawing utils this just basically takes the output similar to what we had up here like we're gonna get um a face mesh detection with a lot of uh different points so let's actually go to Media pipe and look up face mesh um so it estimates 468 3D landmarks so now it's not going to only have the X and Y point but the I guess Z the the depth of the face and um yeah it does a really good job of finding the eyes even the iris and we're gonna see how well it does and then you can use it to do this stuff oh this would be cool to try to do a like a renderer to see if I could do Face Effects like a like we're on Tick Tock or something so let's look at these examples four static images for webcam input so let's go ahead and do this these are all of our inputs so we're opencv this drawing utility drying Styles face mesh which is and then image files I don't think we need this and those are just the specs so yeah for webcam input we'll do this and it's got this different indentation but that's fine our video capture device is seven but we've already created it before so I don't want to keep on creating it and I don't want to release it all righty what do you guys think now this is just like a huge step up from everything that we've done before no don't drink water show me my face mesh so this is the difference between 68 in 2D and 256 I think it's 256. 468 sorry data points in 3D it's pre