Build a Python Facial Recognition App with Tensorflow and Kivy

Key Takeaways

what's happening guys my name is nicole astronaut and welcome to the course in this course we're going to go from a state-of-the-art research paper and go all the way to the end and build a deep learning integrated app which allows you to perform facial recognition using a model that's pretty similar to what you might encounter using face id on your iphone this is going to be broken up into eight different parts where we start off all the way at the start with installing our dependencies completely from scratch to investigating the paper that we're going to be building taking a look at our data building up our deep neural network and then finally taking that deep neural network and integrating it into a qv app so that you can use it in a practical manner ready to do it let's get to it what's happening guys my name is nicholas renate and welcome to the very first episode of papers to code in this series what we're going to be doing is building our very own facial recognition app using deep learning from scratch so this series is all going to be to do with grabbing the data building up the model and implementing it inside of an application ready to do it let's take a deeper look as what we'll be going through so in the very first episode of this bigger series what we're going to be going through is just our general setup so what we'll do is we'll install our dependencies we're going to import all the different tensorflow specifically deep learning components that we're going to need this includes our layers our models and our inputs we're going to also configure our gpu so if you're using colab or if you're using your local machine this is going to ensure that your don't blow out your gpu completely and then what we're going to do is we're going to set up our data folder so there's going to be a bunch of episodes following this but this is really just the setup ready to do it let's get to it alrighty guys so we're going to keep this pretty simple and easy at least for the first episode in the facial recognition or face id type series so what we're going to be doing in this episode is we are going to be installing our dependencies importing our dependencies setting our gpu growth and then creating our folder structures now what i'm hoping is that this sort of episode of this style of video is going to become a little bit more common on the channel so we'll be able to take papers and convert them to code and actually implement them with practical examples so if you like this style of series do let me know in the comments below i'd love to hear your feedback so what we are going to do in this particular case is first up we're going to install our dependencies now the first part of this series is going to be all inside of jupiter so this is all going to be the data science and deep learning side of things where we actually create our facial verification model and we'll talk a little bit more about the paper that we're actually going to be interacting with a little bit later but we'll see that and then once we've gone and done that we'll actually transition and start coding up in vs code when we go and build our facial verification app so first things first let's go on ahead and install our dependencies all righty so those are our dependencies now installed now let's actually take a look at the line of code that we wrote so first up we wrote exclamation mark pip install so this is your standard python install and as per usual we're going to be taking this entire tutorial step by step so again we're going to write the code and explain it so exclamation mark pip install tensorflow equals equals 2.4.1 so this is going to install the tensorflow 2.4.1 version into our environment then we're using or we've imported or installed tensorflow dash gpu equals equals 2.4.1 and if you don't have a gpu on the machine that you're running on there's a few things that you can do about it you can try running this tutorial again without a gpu but it's going to be a little bit slower to train so which is perfectly fine it'll still work if you want to use a gpu there are a whole bunch of cloud platforms out there that leverage jupiter notebook so colab is a free one that you might take a look at but in this case if you don't have a gpu it's perfectly fine then we've installed opencv and so in order to do that written opencv python and this is going to give us some of our image processing capabilities and we're really going to be using this when we actually go to do real-time testing right at the bottom of this tutorial and then we've gone and installed a matplotlib as well cool so all up we've installed four different dependencies so exclamation mark pip install tensorflow tensorflow.gpu 2.4.1 opencv python and matplotlib so those are our four dependencies now installed cool so that is part 1.1 now done now the next thing that we need to do is import our dependencies so we've sort of got two different classes of dependencies that we want to import here so we've got our deep learning stuff so all of our tensorflow components and then we've got some sort of standard dependencies that are more going to help us actually work with our data so let's go ahead and import our standard dependencies first and then what we'll do is we'll import our tensorflow dependencies after that okay those are our standard dependencies now imported so i've written one two three four five lines of code there and one comment so the comment just says we're importing our standard dependencies again it's good practice to comment your code so what i've written is import cv2 so this is going to import opencv into our notebook then we've imported os so this is just an operating system library and we are going to use this when we actually go and create our folder structures so this just abstracts the operating system from our python code so it just makes it a whole heap easier to work with different directories different file paths then we've imported random this one is a little bit optional but i've brought it in just in case so if we are testing generating new data this comes in handy then i've imported numpy so import numpy as np and numpy is a great library when it comes to working with different types of arrays so tensorflow which is the deep learning library we're going to be working with is all to do with different tenses or working with different tenses numpy helps us work with those so what we're then going to do is import matplotlib and numpy is really just a really great array transformation library so again if you need to resize or reshape erase comes in super handy there so we'll probably use it later on then we've imported matplotlib so from map plot lib import pi plot as plt the main reason that i've brought in matplotlib is because there is a method called plot.iamshow so if i'd have employed.i am show this effectively allows us and question my question mark this effectively allows us to visualize an image so you can see that it says display data as an image so it just makes it super easy to visualize it when we're working with our different data components which we're going to need when it comes to facial verification cool so that's done the all right so we've gone and written those five items code so import cv2 import os import random import numpy's mp and then from matplotlib import pi plot as plt cool so those are our standard dependencies are now imported now the next thing that we're going to go ahead and do is import our tensorflow dependencies so i'm just going to change this so we're going to import tensorflow dependencies and really we're going to be using uh well not really we're going to be using 2k oh we need to import two key things here we need to import our model components and the different deep learning layers that we're going to need now what we're actually going to be doing is we're actually going to be using the tensorflow functional api so a lot of tutorials tend to use the sequential api but what i've actually found is the functional api is a lot more flexible when it comes to building really hardcore deep learning models which in this case this one actually is now the model that we're going to be building is called a siamese neural network so this actually allows us to do something called one shot classification we can actually pass through two images and determine whether or not that image is the same so if i actually bring up that paper let's see if i can bring it up and i'll link to this in the description below as well so basically what we're actually going to be doing is we're actually going to be building up this network described in this paper so siamese neural networks will one shot image recognition and if you actually take a look at the architecture here so this is really important so we are going to have two inputs so you can see that there is a break between these hidden layers here so we've got this input and we've got this input what we're effectively doing is we are passing through two images at the top at the same time and then we're going to have this distance layer so this actually measures the similarity between the two images and we're going to train our neural network to determine what that similarity is like so if it is very similar then we're going to output a 1 which basically means it is verified if they are completely different then we're going to output a zero which means we are unverified now in terms of the neural network that we're going to be building we're effectively going to be implementing this or something very very similar to this i think the input shape that we're going to be passing through is a little bit different we'll have 100 by 100 but you will see that towards the end of this series so we're actually going to take this model and actually implement it in real time which is super cool all right we've digressed so this is the model that we're going to be building we're going to be using the tensorflow functional api and i will link to this in the description below but for what we need to do now is we we actually need to import those functional api components so functional api so let's go ahead and do that okay those are our functional api components now imported and we've also imported tensorflow as well so let's take a look and take a step back as to what we've actually gone and imported so what i've written is from tensorflow.krs stop models import model this is probably one of the most important layers that you're going to need so when you're actually defining your model using the functional api what you effectively do is you type your model and you pass through two things so you pass your inputs and then you set that and then you also pass through outputs and you set that so what we're effectively going to be doing is we're going to be building up our neural network and then passing through what we want our inputs to be and our outputs to be so this is a slightly different to the sequential api but it allows you to define a significantly more sophisticated neural network so ideally what we'll actually have is we'll have our input image put this in brackets our input image and then our verification image and what we're effectively going to get out is a layer which effectively outputs one or zero so that is effectively what our model is going to look like by the time we actually get to it so that is what the model class is used for so in order to do that i've written from tensorflow.keras.models import model and this is effectively giving us this and then a big part of our deep learning models are a whole bunch of different layers so in order to do that we've imported a bunch of different layer types so we've imported or we've written from tensorflow.keras.layers import layer so this layer class is a high level class so this actually allows us to define a custom layer now when it actually comes to doing that we can effectively create a whole new class to actually go and generate that layer so it's effectively class um l1 or the class that we're actually going to be implementing is going to be called l1 distance and we can actually pass that layer through so we can perform inheritance for that layer so this effectively allows us to create a custom neural network layer so we'll actually be going through some really cool concepts as part of this bigger series okay so we've imported from tensorflow.layers import layer and then we've actually gone and imported some standard layers so conv 2d this allows us to perform a convolution so if you've heard of convolutional neural networks that is exactly what that allows us to do we've imported dense so this gives us a fully connected layer which is effectively what you typically see in most neural networks that sort of looks like these layers that you're seeing here then imported max pooling so this allows us to pull our layers together and effectively shrinks the information that we've actually got so this is almost like averaging or in this case it's not averaging it's actually taking the max values over a certain region to effectively reduce how much data we're actually passing through to the next layer and then we've got our input so this is again another base class so if you think about the three most important clusters we've got here it's model layer and input so our input allows us to define what we're passing through to our model and our layer effectively or our model actually allows us to compile it all together and our layer gives us our base class and when we're defining our input we're effectively typing input and then shape and you're specifying the shape there and then the last layer that we've imported is flattened so this effectively takes all the information that you've got from a previous layer and flattens it down into a single dimension so that allows us to pass convolutional neural network data to a dense layer so that's what we're going to be using that for and then we've imported tensorflow as tf so import tensorflow as tf so let's quickly recap on what we've imported to from tensorflow.keras.models import model from tensorflow.keras.layers import layer comma conf 2d comma dense comma max pooling 2d comma import comma and then flatten and then we've imported the tensorflow base or the tensorflow class overall or tensorflow overall so import tensorflow as tf so we're going to use this for our image processing and a whole bunch of other helpers but that is section 1.2 now done so what we can actually do is uh delete this because we don't need that and we can delete that so that is section 1.2 now done so what we can now do is go on ahead and set our gpu growth so this you only need to do if you're doing it on a gpu based machine so if you're not doing this on a gpu bet on a machine that has a gpu you can skip this step what we're effectively doing here is we're limiting how much vram tensorflow can use on the gpu because by default tensorflow is going to expand to take it all over and if you don't go and set this then you're effectively going to run into out of memory errors so this helps avoid this so let's write a comment so avoid out of memory errors by setting gpu memory consumption growth right so think of this about like or think of this as like putting some chains around your gpu to let it go to stop tensile from from going crazy or it's more chains around tensorflow than the gpu but effectively we're going to be locking intense flow down so it doesn't go crazy right so let's go ahead and do this oh okay hold on we've uh list fizz okay so that is our memory consumption growth now set so we've gone and written three different lines of code there so first up what we're doing is we're actually accessing all of the different gpus on the actual machine so if i actually type out gpus you can see that we've actually only got one there so if i take a look at the length of that so you can see we've got one gpu so in order to get that we've written gpus equals tf.config.experimental dot list underscore physical underscore devices and then we've passed through a string which says gpu so what this line is doing is it's actually getting all of the different gpus that are available on our machine then we're actually storing that inside of a variable called gpus which is effectively what i'm showing here right so you can see it says physical device name equals forward slash physical underscore device gpu zero so this basically means that we have a gpu here in this case the gpu that i'm using is a rtx 2070 super so i think it's got around about eight gigs of vram so just something to keep in mind sharing some information you know all right then what we're doing is we're actually going through and we're setting our memory growth so we are looping through all of the gpus so again if you have multiple gpus this is going to work as well which is why we do the loop so for gpu in gpus that's effectively just looping through this four gpu in gpus we can effectively print gpu right so we're effectively looping through them but then what we're doing is we're actually setting our memory growth which is this function which you can see right there so i've written tf.config.experimental.set underscore memory underscore growth and then we're passing through the gpu that we're processing at that time so if you remember we're looping through each one of them up here this is actually going to go and set that gpu and set memory growth equal to true got a bit tongue tied there all right cool so you can see that that is now done so we're grabbing all of our gpus we're looping through each one of them if we've got multiple this works and then we're using the set underscore memory underscore growth method to set our gpu growth equal to true and remember we're passing through that gpu and passing through true to actually set that cool so we can delete this cell down here so let's delete that and that is section 1.3 now done now the next thing that we need to do is a little bit of setup so we actually want to set up our folder structure so that we are in a good place when it actually comes to getting our data so if we actually do let me actually show you where i'm working what i'm working with so the folder that we're current i'm currently working in is looks a little bit like this so right now i've just got a virtual environment i've got some jupiter notebook checkpoints and i've got the actual notebook that we're working in at the moment what we need to do is create a couple of folders so we are going to create three specific folders for our data we're going to create one called anchor one called positive and one called negative now let me explain what each one of these is going to do so if we actually take a look at our neural network let's see if there's an example of it okay this is a good example so when we actually go to perform our facial verification we're actually going to pass through two images we're going to pass through an image called an anchor which is think of this as your input image right so this is if we are going and doing it in real time it would be the image that's captured from your iphone or the image that's captured from your webcam or whatever image that is verifying you in real time so think of this as your anchor we then pass through a verification image or a positive or negative image so what we're effectively going to be doing is saying hey does our anchor match the positive image so in this particular case you can see the soccer ball matches the soccer ball so it outputs same when our neural network does it it will output one we then also pass through a our input image again or our anchor image and then we verify it against a negative and we want to make sure that that is different because we effectively want our neural network to be able to distinguish between ourselves and another person so we're going to call the input image and anchor image we are going to call the verification image which is positive the positive image and we're going to call the verification image which is a negative example the negative image now all the negative data we're actually going to get from a repository called labeled faces in the wild which is a sort of like a standard repository but we're going to be doing that in another tutorial i'll show you all in another episode of this series but i'm going to show you how to set all that up so when we actually go to collect our positive and our anchor data we're going to be using our webcam for that so again we're going to be going through that i believe in episode 2 of this cool that is that uh explained so what we now need to do is actually create some folder structures so what we'll first up do is we're gonna set our paths and then we'll actually use the os library to actually go and create those directories so let's set up our paths first up okay so those are our paths set up and we haven't actually created the folders there we're just defining what our paths are going to look like so this is effectively what our directories are going to look like i'm calling them paths but it's just our directories right so i've written pos underscore path and i've just defined all of our paths in negatives because it just makes it a little bit easier to understand what they are so pos underscore path equals os dot path dot join so what os.path.join does is it effectively joins different directories together to be able to form a full file path so what you'll get out of this is data and then it's either going to be a forward slash or a double backslash depending on what type of os you're using so the file path is going to be data and then forward slash or backwards positive so if i actually take a look at that and type out positive path you can see this is just returning the file path that we're about to create and then we've gone and done it for our negative images and our anchor images so remember our positive images are going to be our positive verification images and negative images are going to be our negative verification images and our anchor images are going to do the same now when it actually comes to implementing this on a or on a real life implementation i've got a slightly novel or special implementation that we're actually going to use but again we're going to do that more later on in this series but um we've got something special to make this actually work really really well all right so we've got positive path negative path and anchor path and then what we are going to do now is create those directories so let's go ahead and do that and let's actually just take a look at the folder so we're going to basically create one big folder called data and then inside of there we're going to have three different folders so one called positive one called negative and one called anchor now what we need to do is actually go on ahead and create those so let's go on ahead and do that so i'm just going to delete this cell here and then create a new one let's do it okay so those are our directories now made let's add a comment there make the directories so in order to do that i've written os dot make dears now keep in mind it is m a k e d i r s not mkds uh make dears actually goes and creates all these subsequent folders so if you don't have a top level directory created it's actually going to create the full file path so i've written os dot make dears and then i've passed through our positive path os dot make dears and then pass through our negative path and then os dot make ds and pass through our anchor pass so we're effectively doing this part this path and then this path now if you wanted to you could put this inside of a list and then loop through them i figured it's just as easy to do this but basically we're going to be creating all of our different directories so if we now go into our folder and mine is inside of youtube and face id you can see let me zoom in on that so you can see it a bit better so you can see i've now got this folder called data and inside of that i've got three folders so one called anchor one called negative and one called positive there's nothing in there at the moment but our three folders are created and you can see that we've also gone and created this top level data folder so everything that we we're going to use when it comes to getting our data is going to be in here and then in one of these three repositories cool alrighty and i think on that note that is us done so we've gone and now created our folder structures so we defined our parts and we went and set up our directory so we've gone and done a whole bunch of stuff in this tutorial so we went and installed our dependencies we went and imported them we then went and set our gpu growth and we also went and created our folder structures to get ready to actually start collecting our images and building our face id or facial verification models so on that note that is this tutorial now done see you in the next one thanks so much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a thumbs up hit subscribe and tick that bell and let me know if you've got any questions or comments in the comments below thanks again for tuning in peace what's happening guys welcome to part two in the series on facial recognition using a siamese neural network where we try to implement a code from an existing deep learning research paper and in this particular case we're going to be doing facial recognition or facial verification so ideally we'll be able to use a webcam or a camera to be able to verify ourselves inside of an application let's take a deeper look as to what we'll be going through in this tutorial alrighty so in part two what we're going to be doing is we're going to focus on getting our data now remember from part one what we needed was three different types of data so we needed our negative images we needed our anchor images and we needed our positive images so in order to collect our negative images what we're going to be doing is we're going to be leveraging a standard image repository or facial image repository called labeled faces in the world so we'll be able to download that and unpack it and get it into the structure that we need for our model we're also going to collect our anchor images and our positive images now in this particular case we're going to be using opencv to do that using a webcam but if you've already got some images of yourself say for example you've collected them using your phone you can definitely use those as well ready to do it let's get to it alrighty guys so in this tutorial what we're going to be doing is collecting our data sets from the labeled faces in the wild data set and then we're also going to be collecting our positive and our anchor classes so this basically means that we're going to have all the data that we need to at least go on ahead and train our model now before we get into it i wanted to sort of explain a little bit more as to or how we're going to actually be using these data sets so let's actually take a look at how this is all going to work now as i mentioned before we're going to have a positive data set and we're also going to have a or we're going to have positive examples negative examples and anchor examples so i wanted to sort of visualize how this is actually going to work so let's take a look at our first positive or our positive examples first so say for example we have an input image well that's a little bit thin let's make that a bit bigger so we've got an input image so imagine this is coming from our webcam move this out of the way webcam and then what we're going to have is a positive image so positive when we actually go and build our model what we're going to do is we're going to pass this or pass both of these images to an embedding model or an encoding model it's probably a better term and these models that you can see here so let me change the color so it's a little bit better to see so this model is effectively going to be our encoding model so it's going to convert our webcam or our anchor representation so this webcam data is our anchor so we're going to convert that model encoding to a data representation and then when we actually go and build our model what we're actually going to be doing is trying to determine the difference between our anchor and our positive so this layer that we're going to implement over here is actually going to be a distance layer so think of it as going all right so we're going and converting our input images to a embedding or an encoding and then what we're going to do is we're going to try to see how similar they are and if they are very similar then what our model is going to do is it's going to output a 1 to basically say we are verified which is effectively saying that the person inside of our positive image is the same person inside of our anchor image right so these are all going to be connected now the cool thing about using this particular type of model is that if you wanted to go and implement this on other people then you definitely could all you would need to do is pass through a different positive image and pass through the same anchor image or password different anchor image and it will be able to verify against a whole range of people now in our particular case we're going to be doing it against one person but that's perfectly fine now let's take a look at a negative class right so i'm going to do this one in blue so again actually let's do it in the same color so we're again going to have our anchor image right and this could be from your webcam it could be an image from your phone it's really going to be what we're passing through as our input to effectively perform our verification then we're also going to have a negative example and again we're going to be using this same embedding layer or the same embedding model or encoding model whatever you want to call it and we're going to be passing these images through so these models that you see here or this model is the same across the board i'll actually draw it uh make it a nicer orange that's nice all right so this everything that you see here that is going to be the same model so effectively what that model is going to learn how to do is how to best represent the input images to be able to ensure that when we actually go and perform our similarity analysis that we're actually accurately classifying them as either positive or they match or they don't match as in negative so when we go and pass through our anchor and our negative what's actually going to happen is when we pass this through to our distance layer our distance layer is going to say hey not the same the same and it's going to output a zero so which means that we are unverified so i figured i'd give a little bit of a visual representation of what we're building because sometimes i think it's very easy to get lost as to how these neural network models work but basically this is in a nutshell how this neural network is going to be built up so what we're actually going to be doing in this tutorial is we're going to be focused on collecting our data so this one's going to be so we're going to be collecting our anchors which are there we're going to be collecting our positives which are there and we're also going to be collecting our negatives now in our particular case our anchors are going to come from our webcam so we're going to do that using opencv and our positives are going to come from our webcam as well not wed and our negative data is going to come from the labeled faces in the wild data set so this is an open source data set that actually has a whole bunch of different faces that in a nutshell is what we're going to be doing today we're going to be focused on all of the stuff in i don't know what do you call this color aqua we're going to be doing that so we're going to be collecting that data alrighty so let's actually get back to it and do some coding so first up what we're going to be doing is we're going to be collecting our labeled faces in the wild and then we're going to collect our positive and anchor classes so let's go on ahead and do this now in order to get our labeled faces in the wild data set you can actually go to this link here so let me actually copy this so you can see it so it is http colon forward slash forward slash vis www.cs.umass.edu forward slash lfw forward slash so this is actually going i'll actually link to this in the description below so you can pick that up as well so don't stress if you haven't picked it up now as per usual all the code that we write inside of this tutorial is going to be available via github so if you want to pick that up you definitely can and i'm actually structuring the code so you can see what we've written after tutorial one or part one of this series what we've done after part two so you'll actually be able to see progression okay but for now what we need to do is get our labeled faces in the wild data set so if we go to this link what you'll actually see is that right over here we've got this link called download so we're going to hit download and then there's a whole bunch of information here so we've got uh what do we have so we've got all images as gziptar file all images aligned with deep funneling all images aligned with funneling all images aligned with commercial face alignment software so there's a whole bunch but what we're actually going to be doing is we're going to be using this one here so all images as a gzipped tar file so let's hit that and this should start downloading so it's about 170 273 megabytes so once you've got that we'll be able to untie it and start working with it inside of python so let's give that a second to download alrighty so that is our data set now downloaded so what we're going to do is we're going to open that up inside of its folder let's zoom out a little all right so you can see that that's downloaded so i'm just going to cut that and paste it into the same folder that we're currently working in so i am currently inside of my d drive inside of youtube and inside of our face id folder so i'm just going to paste that there so once you've gone and downloaded it put it or once you've gone and downloaded that data set grab it and put it inside of the same folder as your jupyter notebook is in so if you're doing this in colab just make sure it's in the same folder as your jupyter notebook so you can see that that is our jupyter notebook that we're working on at the moment that is our data set so lfw.tgz cool alrighty so what we now need to do is we now need to uncompress that so it's a tar gz file so you can see it's tgz so we need to uncompress that so we can go on ahead and do that inside of our notebook so i'm just going to add a comment uncompress gz what is it labeled faces in the wild data set all right so we're going to let's do it okay so that is the command to uncompress and extract our data set so i've written exclamation mark tar and then there's a space dash xf and then there's a space and then there's the name of the file so this command here is what's actually going to allow us to extract our data set and put it inside of the same repository or the same place that it currently is this is actually just passing through the actual name of the data set so if say for example in the future the name of the data set from labeled faces in the world changes you're going to want to change this component here so if we run this now all things holding equal this should uncompress and we should be able to see our data set okay so that's finished running so you can see that we no longer have an asterisk over here so if we actually go and open up our data set again i don't know why i closed the folder that's cool you can see that that's now been uncompressed so we've got this folder here called lfw and we have a ton of images so you can see that it's actually labeled by person's name now we don't we're not actually really concerned with the person's name in this particular case because we're going to be using all of these for negatives but if you wanted to do a different form of facial verifications to say for example you wanted to add triplet loss you could definitely do this and add pairs of images for different people in our particular case we're very much focused on the one person that we want to verify so what we need to do is we need to take all of these images so if you actually open up these folders there's multiple pictures of people right heaps of people list keeps going what we want to do is we want to take these images inside of the labeled faces in the wild folder and inside of these subsequent folders and we want to put it inside of the folders that we created in part one of this tutorial so if you go into the data folder that we created earlier we want to move all of those images from the labeled faces in the wild data set and put it inside of this negative folder so what we're going to do is we're going to write the python code to go on ahead and do that so this is effectively going to put all of our data in the same place and follow the same structure so let's go on ahead and do it so we're going to i'm just going to add a comment so move lfw images to the following repository that's going to be data and then forward slash negative all right so let's go ahead and write that code okay so that is our code to move our data from the lfw folders and the subdirectories into our negative folder now the key thing that i just realized is that i haven't actually gone and run the code that we had in our initial tutorial so if i go and run this again so let's actually take a look at what we've written first and then we're going to go and run our imports and stuff so i haven't actually gone and run the initial steps my bad that's fine um so if i actually go and run this now we're going to get a whole bunch of errors if i run this you can see it's saying name os is not defined perfectly fine we'll solve that in a second so first up what we're doing is we're looping through every single directory inside of our labeled folders in the wild repository or directory then what we're going to do is we're going to loop through every single file inside of those subdirectories so we're effectively saying go into ow where are we surface id so go into this folder and then go into this folder and loop through each of these images because in some particular cases there's multiple images of people so you can see in that case we've got multiple images so we need to move each image into its new folder so in order to do that we first up define the existing path so i've written x underscore path in caps and i've set that equal to os the path dot join and then we've passed through lfw which is the main directory or the root directory pass through the directory that we've extracted from here because remember we're looping through them and then pass through the file name that we're getting from here then we're specifying the new path name so i've written new underscore path in all caps and i've set that equal to os.path.join and then we're passing through a negative path which is from our previous tutorial which we defined over here and we're passing through our file name so this is going to effectively join our negative path and our file name and then we're going to use os dot replace and we're going to pass through our existing path and our new path so this is going to grab it from our existing path and move it to our new path but as of right now this isn't going to run because we haven't run our import so i'm going to go right up to 1.2 run this so that's going to import open cv os random numpy and matplotlib then we'll import our tensorflow dependencies not the i think we'll need them now and then we're just gonna run uh the code under 1.3 and we're going to run the code under 1.4 so that's going to define our different paths our positive path our negative path and our anchor path okay now if we go and run this all things holding equal let me actually just show you quickly how this actually works so if i write os dot list here lfw so this actually returns all the subsequent folders inside of the lfw directory now if i loop through those so for directory in os.listia what's going to happen is i'm going to be able to access each directory so if i write for file in os.list dr and this should be os.path.join i've actually written got an error there so os stop let me actually run it without changing it and we'll see what happens so if i run lfw and then directory that's going to throw an error so this should actually be os.path.join because right now i'm not joining those directories together so if i change that here as well os.path dot join and then if i print a file this is going to print out every single image so now if i actually join these together so os.path.join and if we pass through lfw so os.path.join just joins directory names together so it gives us a full path so if i pass through lfw and then directory and then file and then close that this is going to give us the full path to every single image right so that is exactly what we're doing to get our existing path then what we're doing is we're defining the new path so we're going to do os.path.join and we are going to be passing through our negative path and our file right so we're effectively going to be grabbing this image and then moving it into data and then negative and then aaron eckhart and so effectively it's the same name so we're grabbing this and moving it to here we're doing this we're moving it here so we're just going to loop through and do this for every single image so if i delete that we don't need that anymore and i actually go on ahead and run this this is actually going to move all of our images from those existing stacked directories into our negative path and that is done so it ran reasonably quickly so if we go into that folder now and go into d drive youtube face id if we go into let me zoom in on this data and then negative you can see we've got all of our negative images there pretty good right now i don't think we're going to use all these images for training but you sort of get the idea we've got plenty to work with if we need to so what's happening now so we can actually close this so what we just went and did there so if we go into our lfw folder and you can see that each one of these are now empty because we've actually gone and moved them into a new folder so what we can actually do is delete this lfw folder there's nothing in it we can get rid of it now so if we delete that we are all good cool so that is step 2.1 now done so we've gone and uncompressed our label faces in the wild data set and we have also gone and moved all of those images from the lfw directories in to our negative path which we defined up here from the first episode in this series cool so what's next is that we need to actually go and collect our positive and our anchor classes now for this what we're going to be doing is we're going to be using opencv to access our webcam and we're going to collect those images down and save them down now the size of the images that we're actually going to be collecting are going to be 250 pixels by pixels so by default when we use our webcam your image resolution might be a little bit different so what we actually want to do is we want to ensure that we're collecting images of that size because i believe the sizes from the lfw data set are going to be in that same size as well so let me just double check that so if we go into data and then negative and if i go and open one of these so properties and details yep so you can see that 250 by 250 and i can check another and again 250 by 250. so to make our lives a little bit easier we're just going to ensure that we collect the same or images of the exact same dimensions when we collect our anchor and our positives this is going to make your data processing a whole bunch easier when it comes to training the model all right cool so that is good so what we now need to do is do exactly that so we're going to be using a pretty standard opencv loop to be able to go and collect this with a few tweaks to make our lives a little bit easier but first of what we need to do is ensure we can access our webcam and do that successfully so let's go ahead and do that okay so that is the first part of our video capture code or image capture code now done so i've gone and written one two three four five six seven eight different lines of code there so this is if you've watched any of my computer vision tutorials this is going to look super familiar to you so let's actually comment through this so first up what we do is we establish a connection to the webcam and that is exactly what this line is doing here so i've written cap equals cv2 dot video capture and then i've passed through a video capture device now i think it's going to be video capture device 3 but it might be a little bit different because i've gone and installed some new stuff on my pc but we'll see and then i've ridden so i've actually i think it's actually because i'm actually doing the whiteboarding now but and let me know what you guys thought of that i'm testing that out we'll see if it picks up or if you guys enjoy it if you don't let me know and i'll stop doing it so once we've established our connection to the webcam i've then written while cap is open so this is going to loop through every single frame in our webcam and then we're using cap.read to actually read that capture at a point in time and then what we do is we actually unpack the results that we get from that method there so we unpack it and get a return value and the actual frame so this frame is the actual image then what we're doing is we're rendering that image back to the screen so it just makes it a little bit easier to actually see what we're doing so we've written cb2 dot i'm show so let me add some comments here so show image back to screen so cv2 dot i am show and then we're naming what we want our frame to be named so in this case every image collection can name it whatever you want and then we've gone and passed through our frame which is what we got from over here so this is effectively going to be showing the feed from our webcam on our screen inside of python or inside of a cv2 frame and then everything from here on out is to do with breaking gracefully so what we've written is if cv2 dot weight key one so this is going to wait i believe it's one millisecond so cv2.weight key uh so it says delay what is it uh this function yes it's in milliseconds so it's going to wait one millisecond and it's also going to check what key we've actually pressed so this is actually unpacking what is being pressed from our keyboard so then we've written end zero x f f equals ord q or equals equal so let's check it's doing a comparison check this is really important because we're going to use this a little bit more in a second so when we hit q on our keyboard this should close down our frame what we're also going to do in a second is we're actually going to configure some other ones so that when we hit a it's going to collect an anchor and when we hit p it's going to collect a positive image and i think we're going to collect roughly roughly 300 issue images doesn't matter we'll see i think 300 is probably a good good starting point okay so that is doing that check and then if that check is passed so if it waits a millisecond and we hit q on our keyboard then it is going to break out of this loop up here and then it's actually going to release our webcam so let's actually comment this for once release the webcam and then it's going to destroy or close the image show frame so if ever you are using opencv and you're accessing your webcam and stuff is just freezing up it's not working what you can actually do is run cap.release to release your webcam and then cv2.destroyorwindows to actually close everything down and re-kick things off so if this number up here is incorrect and it all locks up and freezes up what we'll do is we'll stop this cell from running and then we'll run these two commands down here to be able to release whatever webcam we're trying to access and then destroy all those windows so that is uh image collect well that is our webcam code now set up we haven't actually done any image collection yet so if we go and test this out let's see if that works so if that does run successfully you'll get a little pop-up right so that has not run successfully so this error is a common error that i always get asked about so error opencv 4.5.3 blah blah blah blah source dot empty in function so this basically means this here super important let me zoom in on that that basically means that it's not able to access the webcam so whatever we're getting back from that webcam device or that image device is empty which means that we don't have the right webcam number up here so what we want to try to do is try a different webcam number so let's try four that might work a little bit better all right so that's worked so you can see that i've got this little pop-up and you can see our feed all right so sometimes you're gonna have to tweak that right so and this is it's great that i'm showing you this because it'll actually show you how to resolve that error so all we did there is we change our video capture device from three to four and you can see that i've now got the right video capture devices i can see myself in the screen and you can see up there that it says image collection so that image collection label is coming from over here so if we wanted to change that you definitely could okay so the key thing now is that this video frame that we're currently looking at is not