Unbelievable Face Swapping with 5 Lines Code

Key Takeaways

recently I was scrolling through github's trending python packages as one does and I came across a library that looked pretty cool called Insight face and it's for facial recognition now I've done a video before on this channel about deep face which at the time was a state-of-the-art facial recognition python library and this one was getting so popular that I figured I'd give it a try not only does this package do facial detection and Analysis but the thing that really drew me towards it was its ability to do face swapping I'm sure we've all seen face swaps made in Photoshop where you essentially just cut out the person's face and stick it on someone else's but this type of face swapping is not like that it's really amazing so I figured I'd make a short video to show you the code that I wrote In order to get this to work and if anything this video is to show you how you can find open source packages that maybe Implement something that you're interested in and test them out on your local machine and give them a try okay let's start coding here's the website for insight face you can see they link directly to their GitHub page they explain basically why they're the best in the business and they've won some awards that sort of back that up so the first thing I did was took a look at their repo and you could see that they have some documentation that goes along with this code as well as some examples in a quick start guide now I'm interested in writing code in Python so we're going to be using their python package but there was a little bit of a hiccup that I ran into when I was trying to recreate this tutorial that they have and the next thing I did was created a new conda environment where I pip installed the Insight face package I like doing this for new projects because it keeps my environment completely isolated from all the other environments that I'm working with so just to test out this package I created a jupyter notebook and I made sure that I activated the Insight conda environment that I had installed inside face now I'm importing some packages that we'll use here CV2 and matplotlib can be used to read image files and to display them and then of course we're importing Insight face which is the package that we'll use to do the face analysis insightface has this app object for face analysis and this Insight git image is just a helper function that they created for downloading an example image just so you know what package versions I'm running I'll print the versions here for you and the process for doing this face swapping actually takes part in Two Steps step number one is detecting the faces and step number two is doing the swapping in order to detect the faces we're going to use this face analysis object that we imported and we're providing it with the name of the face detection model that we want to use in their documentation they go over the different different types of models that you can use and the one I'm using Buffalo L is one of the larger models so when you run this the first time it will actually download the model weights into this dot Insight face folder in your home directory and let's go ahead and test this out so I'm going to use this Insight get image function just to get their example image from their tutorials and I'm using matplotlib here to display the image now I'm taking the last channels here and I'm reversing the order because matplotlib actually plots channels in a different order than CV2 which was used to read this file I have a whole video about working with image data that goes into detail about that but we can see here this is just an image of all the characters from the TV show Friends and we're going to do some face swapping with this image so let's take this detection app that we created and let's run a DOT get on it and provide this our image this will detect the faces we do get some warnings here but it does run fine but if we look at what this returns it's actually a list of dictionaries and each item in that list is a dictionary for a face that it's detected we can even take a look at the length of the faces and we see that it's six which matches the number of faces we see here so let's take a look at one of the items in the faces and if we look at just the keys to this dictionary we can see we have the bounding box location of the face some key points the detection strength and also a bunch of landmarks as well as predictions for the gender the age and embeddings now these embeddings are going to be used when doing the face swap so it's important that we've run using a good detector to get the detection but just to make sure that it's cropped the correct faces let's go ahead and plot a crop of each face I'll make here a matplotlib subplot with six plots one for each face and we're we're going to fill this in with the faces so I wrote this code that essentially takes these faces and we enumerate over them then we take the bounding Box locations and slice our main image so that we could see what it's detected and you can see it's each of the Six Faces in our main image so this looks good to go we're almost ready to do some face swapping for step two we're gonna do the face swapping in order to do this we need to use Insight faces model zoo and we're going to use this in swapper model this model has a Onyx file that has the model weights and by default this tries to download the model weights but they've removed them luckily for me I found a Reddit post where someone had saved off these weights and I was able to download them I'll link that in the description if you want to give that a try so we can see now we've created our swapper object so let's choose one of these faces that we want to swap for the others so I'm going to take the first face the that we have here and make that our source face and if I plot the first face we can see it's Ross it's Ross's face that will be swapping in for all the others and now we're going to use the swapper so we have our swapper object and we're going to use dot get and Dot get takes the image that we want to get the face from we provide it the target face and the source face and then we're going to actually use paste back equals to True which means it's going to paste over in our image with that new face but we don't want to run this on our original image because we want to keep that intact so I'm going to make a copy of that image and just call it res this is the copy of the image that we're going to use and swap the faces on and we're going to want to do this six times so we're going to say for face in faces and then we're going to run swapper get on this image swapping the original face with the source face that didn't take long at all to run and let's take a look at the swap faces and just like that now we have our image with Ross's face taking over everyone else's just to show you that a little bit more clearly I wrote this code that Loops through each of them and plots them individually I just think it's interesting how his mouth expression is definitely different it even accounts for the face being turned to the side so of course I couldn't let it end there so let's try to do some swapping with my face so when I was doing the live stream I took a screenshot of my face which we'll just call Rob and to show what it looks like when I took the picture now we're going to detect my face from this image and swap it out with all of the friends faces and the steps to do this are just like before I'm detecting my face in the picture that I took I'm copying the original friend's image and then I'm swapping out my face and pasting that back over that image finally here I wrote some code that'll just plot this result and let's see what it looks like oh that's right I need to make Rob face equal to Rob faces just the first one and there we go my face has been swapped out for each of the faces and friends here and of course this doesn't only work for this picture of friends I took this Monty Python image and did the exact same thing to it and with just these few lines of code we're getting the faces from the Monty Python and swapping them out with my face and there we go it's actually swapped out my faces it even has the mouth open here and I don't know tell me what you think about how well it deals with the fact that there's a beard of this guy here in the center the next thing I did was created a function that would do everything in our process from before but make it a little bit more reproducible so I called this function Swap and show and you provided two image file names and then I have these variables that will plot before or after depending on if you have these as true or false this code is very similar to what we just did above it just assumes that there's only one face and it does the swapping to both images with the opposite face so to show how this works Let's test this on two sort of famous people in the machine learning industry and we're gonna swap their faces and see what it looks like I'm going to leave the plotting before and after shown so you can kind of see what the difference is all right here's the before picture and the after picture I also have the function returning the two images which I don't even want to show so I'm going to make that sent to an underscore that's just how you throw away the result of a function in Python now I'm surprised that this even handles the glass as well running it on Obi-Wan and Spock not too bad not too bad so next I wanted to swap faces of two people in the same image I just wrote a function that does it very similar to the last step but it assumed means that there are two faces in a single image and then does the swap now I called this function Swap and show same image and it only takes in a single image and the app and swapper objects I thought it'd be fun to test this out on Celebrity couples so I have four different couples pulled in here and I'm going to pull it through our Swap and show same image function that one's interesting okay before and after and finally so there you go I showed you how I used an open source python library to swap faces and you should be able to do it just as easily too if you've watched until this point hopefully you enjoyed this video so give it a like subscribe and send this to 20 of your favorite friends that you want to swap your face with let me know in the comments if you want to see anything specific in the future I'll see you all next time