Easy Face Recognition Tutorial With JavaScript

Key Takeaways

are you tired of going to watch your favorite show with one of your friends but they don't know who any of the characters are so they completely ruin it by asking you who's that who's that oh who are they the entire time well no longer will that be a problem because in today's video we're gonna look at creating a facial recognition app where you can upload any photo you want and it'll tell you exactly who's in that photo let's get started now let's take a quick look at the project we're going to build what we need to do is select an image here and we could just take any image we want and then it's going to find all the faces in that image and give us the name of all the people for the faces that it recognizes to get started doing this I already have quite a few files in our project as you can see I have a folder here with labelled images this is just facial images of all the different characters that we're going to be able to put labels to their names so we have Black Widow Captain America and so on and these are just face shots of all these different people you can find these yourself if you want or you can find them on the github that I have linked in the description below and then also besides these labelled images I have the various different models that we're going to need this is what our AI network is going to use to determine people's faces as well as recognize their faces from these images and these again are going to be in my github repo link below so you don't have to go worry about finding them yourself then lastly we just have a couple test images here these are just the various images that we're going to use that have these people's faces in them and again they're in the repo linked below lastly I have the face API minimize library here which we're going to be using to do the face recognition inside this project and all we need to do is add a new file here which is going to be our index.html this is going to be for rendering our page and we're also going to have a script jeaious which is going to be all of our custom code that we're writing for recognizing these faces now to get started let's set up our index.html we could just hit exclamation point enter using Emmet in vs code and that will automatically generate this template for us and inside of here we can just say face recognition because that's what we're going to be building and inside the body it's going to be super straightforward we just need a single input which is going to be a type of file and we just want to give it an ID so we can reference this in our JavaScript we're just going to say image upload and then I'm going to use a few Styles here inside of the body or inside of the head tag because I just want to build a style our page that everything is centered and we'll just select the body we're going to get rid of all the margin all the padding we want to make the width here equal to 100 view widths so it's going to take up the entire width and we want to make the height equal to 100 view height which again we'll make up take up the entire height we want to use display flex combined with justify content center and the line items Center and this is just going to send our content horizontally and vertically then lastly we're just going to use flex direction and we're going to make this column and this way our different elements stack on top of each other as you can see our selector and our image are stacked on top of each other instead of side by side now lastly we're going to be using a canvas element to draw these different rectangles around the images of our characters so we need to absolutely position this so it goes right over top of our different image so we can just make this here a position of absolute make sure I spell that correctly there we go and now we can jump over into the JavaScript which is going to be worth of really interesting parts of this project come into play and if you haven't already seen my facial detection video which uses the same exact library I highly recommend you check that out now if you has certain parts of what we're doing here are what we covered in that first video and I don't want to cover them as in-depth in this video so make sure you check out first if you haven't already now I'm moving on what we want to do is first get our image element so we get our image upload we're going to be equal to document dot get element by ID and we just want to get the ID here of image upload which is the ID that we gave our element and this is so that we can get the files that were uploading next we want to load the different models that we're going to be using and the various models that we're going to be using we're going to use face API Nets dot and then in here we put the library name that we're using so we're gonna use face recognition because we want to be able to recognize the faces so face recognition net and we want to just load this so load from URI and we're loading this from our models folder so we just passed it in here a path to our models folder we need to do this three times because we are using three different libraries here we also have face landmark sixty-eight net and essentially this allows our algorithm here it's a detect where the actual characters faces are so we can use the recognition to find out which face is which and then lastly if we need to use SSD mobile net v1 and essentially what this is going to do is it allows our algorithm to determine which ones are phases this is just our detection algorithm right here and we want to do all these asynchronously so we can just say promise dot all because these all return promises throw them inside of here and make sure we put commas at the end of them because this is going to be an array of elements and then in here when we finish loading all of these we want to call a method which we're just going to call start so let's create that function down here start just like that in here is going to be all the code that we run after we load all of our different models and for now the only thing we're going to do in this start method is we're just gonna take our document body we just want to append a single element here which is just going to say loaded this is just so we know when our page is finished loading all these different models and now let's actually run this using live server so we'll open this up here off to the side and you'll see we get our upload image right here for choosing our file and you'll notice our loaded text is not showing up and that's because we need to make sure we include our different libraries inside of our index.html so we can get our script tag here we want to make sure we defer it so it loads after our body is loaded and we want to first load our face API library and then after that we want to load our script tag right there let's save that and now we should see over here we see loaded pops up as soon as it's done loading everything and luckily loading those models is actually fairly quick now back into our script is let's actually configure this start so we can start recognizing faces using face detection in order to get started doing that the first thing we need to do is take our image upload here and we want to add an event listener for whenever we are changing our element so whenever we actually select an image we want to call the code in here we're going to make sure this is an asynchronous function because we're going to do a bunch of asynchronous code using the face API library inside of this function now as soon as our image is loaded the first thing we want to do is get our image so we're just going to create a variable here we're going to say image and it's going to be equal to our face API whoops face API dot buffer to image and we're going to take the image upload and we want to get the files from it and we just want to get the first one since we're only ever uploading one file and we want to make sure we await this because it's asynchronous and essentially what this is going to do is it's going to take the file that we uploaded and converted to an actual image element which is going to be usable with the face API library next we can use that to get our detections so we can create a variable here called detections again we're going to await our face API and what we want to do is we want to detect all faces just like this and we pass in the image we want to detect the faces from which is the image that we uploaded then we want to do is we want to make sure that we're getting this with face landmarks and this is going to allow our algorithm to determine where the different faces are and we also want to get it with face descriptors which is going to allow us to draw the boxes and such around our various faces of our characters and now to make sure this is working we can just take the document oops document dot body dot append and we just want to append our detections and dot length to see how many different detection zhh we have now let's save this and as soon as it gets loaded over here let's select an image for example this one right here and it's going to upload that image and you can see it found six different faces inside that image but one thing you will notice is we're not actually displaying the image that the user selected so let's work on doing that now this is really easy to do actually all we need to do is in here we just want our hopes document dot append our image make sure we do that on the body and now if we save this wait for it to load select an image you'll see that the very first thing it's going to do after it gets that image from the buffer to image is it's going to actually upload that image here so that we can see it and if we scroll down you can see that it says there are six faces and if we look at this we have one two three four five six different faces in this image so we know that this is working perfectly fine next let's actually work on drawing rectangles around the different faces of our character and the first thing we need to do that is to actually overlay a canvas on top of this image and to do that we need to put them inside of a position:relative element so let's do that now which create a container we're just gonna call it container and we just want to set that equal to document dock whoops the document create element and we just want to create a simple div and we also want to take the style of this container and we want to make sure we take the position we want to set this to relative so that way we can absolutely position our container inside of this and now instead of actually appending here our loaded we can just call this and we can upend our container as well to our body just like this and now we're going to have that extra div being added to the body of our element as well as the text loaded now with that container being added let's work on creating our canvas and adding that as well this is going to be really easy we could just do const canvas here we want to set that equal to our face api create canvas from media just like this and we're just going to pass in the image and this is going to create a canvas from the image that we passed to it and we also want to make sure we resize this canvas to the correct size so first let's add it we're just going to say document dot body dot append and we want to append our canvas here and actually instead of appending to the body what we want to do is we make sure it would pend it to that container so we want to append both our image and our canvas to that container so that way they're both absolutely positioned on top of each other perfectly inside this relative container here and then the next thing we need to do is we want to get the display size of our different image right here so we're just going to say our width is going to be equal to image dot width and we're going to say our height is equal to image dot height make sure I spell this properly just like that and now we can resize our canvas so to do that we can just say face API dot match whoops spell that properly match dimensions and we just want to pass in here our canvas element as well as our display size whoops display size just like that there we go and that's actually going to resize our canvas to the exact same size as our image which is perfect now we can display these detection inside of our canvas now we can display these detection to the user so let's remove this count since we no longer want to display this and instead what we want to do is we want to resize our detections so we're going to create a variable here called resized detections and we want to set this equal to our face API dot resize results just like this and we're going to pass in our detection and what this is going to do is it's going to resize all our boxes for our detection to be the correct size based on the sizes that we pass it and in our case we're going to just pass it here our display size and there we go our detection zarnow going to be scaled to our image to fit in our canvas properly now in order to draw that actual box we want to you as we just want to create a variable which purrs you're gonna call it draw box we're going to set this equal to a new face API whoops face API draw dot draw box and this is going to allow us to draw our box onto the screen and we first want to pass at the box which in our case is just going to be from our resize detections so we're going to use a loop here we're going to say we're going to do for each one of these detection z' detections there we go so for each one of our detection so we want to draw our box and our box here whoops Const box is just going to come from our detection detection dot detection dot box so this is going to be the box of the face that our algorithm determines for our individual faces in our image and we're going to first pass that to our drop box and then we're just going to put a label on it as well so we're gonna say label and for now we're just going to place hold this as face up from here this is where we're going to put the character's name later when we work on face recognition and now we just want to draw that box so we can just say draw a box dot draw and we pass it the canvas we want to draw on to now let's save that wait for it to load over here click Choose file we're gonna choose this file and as soon as everything's unloading this should pop up and we should see boxes around all of our different faces and you will notice that instead of our box is being shown up over the faces they're showing up all the way over here on the right and that's because when we position our element absolutely in our index we need to tell it we want it positioned with the top 0 and the left of 0 so that it's going to show up properly overlaid over our image so now again let's go back here it's loaded select our image wait for it to do its final loading and processing and as soon as that's done it'll pop up and then we should see our face rectangles show up over top of the faces of all of our different characters with the label face and as you can see it's done that perfectly so now let's actually move on to the recognition portion where we can put the different names of these different characters instead of just the generic text face in order to do that we need to parse all the different labelled images that we have here in this labeled images folder so to do that we're gonna create a simple function just put it all the way down here we're going to give this function here a name of load labeled images make sure I spell that correctly there we go and it's just not going to take any parameters and inside of here we're going to do all the loading for our images and if we open up over here are labeled images the first thing we need to do is to get a name of all the different labels that we have and this is just going to be just really straightforward we have black widow these are just the names of a folders that we have we have captain oops Captain America we have Captain Marvel Hawkeye whoops make sure that all the capitalization is the correct for all these Jim Rhodes and Thor and lastly we got Tony Stark and you will notice that I don't have all the different Marvel characters in here this is just a demonstration of just a few of the characters so that we don't have to wait here for hours for me typing out all their names and the next thing that we need to do is we're going to create a promise so we're going to say return promise all because we're going to create a bunch of different promises to load all these different images and detect the faces in them so what we want to do is inside this promises dot all we want to go through all of our different labels so we're just going to say labels dot map this is going to go through all of our labels and return an array for us and we want to make sure this is an asynchronous function and each one of these is going to have our label so there we go now we have a function inside here with all of our individual labels so Black Widow Captain America and so on and inside of here we want to create a simple for loop and this for loop is just going to go from 1 to 0 so we're going to say I equals 1 we're gonna go I is less than or equal to 2 I plus plus and the reason for this for loop is because I have two images of every single character you can create more images or less images but the more you have the more accurate the face recognition is going to be so now that we have this loop and set it here what we want to do is actually load our image so we're just going to say Const image is going to be equal to awaiting the face API again but this time it's going to be calling a method called fetch image and in here whoops just fessed image and in here we're going to pass our image text for the path to where our image is at but one thing to note is that live server doesn't actually serve the images inside of our folder so we can't just do something like slash labeled images slash name and so on may actually need something on a website and luckily I have all these hosted on my github so we can use this URL here which essentially is just HTTP slash slash raw github user content com slash web dev simplified slash the name of the repo which is face recognition JavaScript the master branch and labeled images and inside of here is going to be exactly the same as this folder that you can see on the left here if you want to use your own folders and labels you're gonna have to upload them somewhere on the internet so that your computer can access them using this fetch image and now the next thing we need to do is actually put the label so we're just gonna in here put label this is going to be the name of the folder we want to access and then we need to know which one of these folders images we want to access which in our case is either going to be one or two based on this I variable and all of these are JPEGs so we can just say jpg at the end here and now this is going to load all of our different images into these different variables and we want to do is we want to actually detect the faces in those images so we can just say Protection's here is going to be equal to a weight face api dot detect single face because we just have one face in all these different images and we're going to pass in the image and again we want to do this with face landmarks and we want to do it with face detection or with face descriptors that we go and now once we have all those detection x' we want to add them to an array so let's create that everything we're just going to call this here descriptions we're going to initialize it to an empty array and then down here we can take that descriptions and we want to push in our detection dot descriptor whoops descriptor just like that and this is going to be what describes the face that was detected inside each one of our individual images and now once we're breaking out of this loop right here afterwards we just want to return here a new face api dot labeled face descriptor whoops descriptors and in here we're going to pass it a label as well as the different descriptions that we created to for each one of our different images and then this all of this is going to return all of these labels inside of a bunch of different promises so when we use this we need to make sure we're doing it asynchronously so this is going to be an async function now for our start and what we want to do right before we do our as we want to create a variable here which is going to be called labeled descriptors next from here we can create what's called a face matcher so we'll say face matcher we're going to set that equal to here a new face API a Smasher and instead of here whoops make sure this is capitalized m and inside of this face match sure we want to pass and first are labeled face descriptors as well as the percentage value we need in order to recognize this as a face so 60% means that the algorithm is 60% sure that this is who that character is so as long as the value is above 60% we say okay that's fine we're going to accept that this is who the character is and 60% is a pretty good value here to use now what we want to do is we want to use this face matcher down here with our results in order to actually display the character's name so to do that we're going to create a new variable called results we're going to set that equal to our resize detection dot math and we're going to change the detection so we're just going to use D for detection and we'll use that face metric here and we want to find the best result so find the best match and we want to find the best match from the D descriptor and what this is going to do is it's going to go through all the images that we loaded and it's going to find the best match that is above 60% so is there's none above 60% it's going to return nothing but as long as we have at least one over 60% it's going to choose the highest percentage value that's the closest so then that's going to give us all of our results here so now we can come down here and loop over our different results instead and instead of here we're going to have our result as well as the index of that result because we need that to access our resized detections and then we can change this here this is going to be our resized detections of the index of AI and this is just because we no longer have access to that single detection because we're inside of a different loop and now our label down here is just going to be result dot to string just like that and now if we save this and we wait for it to load and like I said this is going to take significantly longer now because we have to load all these different labels but as soon as all these reloaded once so as soon as your page loads that initial time every subsequent image that you tried to recognize faces in is going to be quick and you don't have to do this loading every single time that you do an image recognition and you will notice that it actually load and that's because down here we called with face descriptors and it should just be with face descriptor it's not saved that so it's going to actually start reloading and again it's going to take a little bit of time for this load to text to show up because it has to parse all the different loaded images that we have inside of our folder and there you go we see it just finished loading so let's choose that same number one image that we've been using and you can see that it recognized Thor it doesn't recognize Ironman because we don't have any images for him recognized as Captain America Black Widow Hawkeye and of course it doesn't recognize Hulk because we don't have any images for him either and now if we wanted to change images here to a different images you're gonna notice one thing is wrong it doesn't actually remove our image and it shows our things over top of each other so we need to actually clean up our code every single time we change images by removing the old ones luckily this is actually really easy to do all we need to do is take our image in canvas variable and pull them outside so we're just gonna say let image and we're gonna say what canvas this is just going to create two undefined variables of image in canvas and then the very first thing we need to do is try to delete them so what we want to do is we just want to come up here say image dot remove whoops image dot remove and this is going to remove the image but we only want to do this if the image actually exists so we're just going to say if image and do the exact same thing with our canvas so if the canvas exists we want to remove it now let's save that and wait for it to load again and now that's done loading let's open up our initial image that we've always been using you can see that that works fine and now if we change it to for example this image you'll see all the old stuff gets deleted and the new stuff gets propagated onto here perfectly and that's all it takes to do really simple face recognition if you're interested in my face detection video and I haven't already seen it make sure to check that out linked over here also subscribe to the channel for more content just like this thank you very much for watching and have a good day