How to make AI Faces. ControlNet Faces Tutorial.

Key Takeaways

Hello friends let me show you how you can control faces inside of stable Fusion you can have an input like this and output results like this and this and this and this you know how it goes with control net and this and this let me show you exactly how it works and I got a couple of tricks up my sleeve to make sure that it's going to work out well for you let's get started by the way do you know what kind of flowers grow on your face two lips if you're new here and need to install control ad check my previous video on that I'm also gonna link it in top right corner right now if you're on computer if you're on mobile you're probably gonna have to check it in the video description alright so I've loaded up stable Fusion here I'm gonna drop an image into control net here which is in a woman shouting that I just now generated and we're going to enable control net I'm gonna set the preprocessor here there are a couple to choose from when it comes to the face and getting the face to work and let me quickly show you so you have the face here the face only potential also the full depending on on your image but for this particular image we're going to use the face and the face only so let's start with the face and let me show you if you click allow preview here and press the little uh explosion down here you can see a preview of the preprocessor so as you can see here you have dots going outside on the outline of the face on the mouth the nose and around the eyes so here you can control the pose of a face now the difference between face and face only now that is the little lines here and that indicates well the direction of the head and the upper torso or you know the the shoulders here so if I would set this to face only for example I would not get those lines I will only get the face and you might ask well what what does that mean what can I do with that if you use this then the body can take any shape around this face if you have it at this then the body is restricted to having this pose it's a little depending on what you want if you're using full here you would get the same result because it's basically a full body with the face it really doesn't matter if you're using full or face if you just have a portrait for the model here you're going to want to use control version 1.1 and then here the open post now we are using stable Fusion 1.5 models and this is the correlating model in control net there are 2.1 models out there so you can find those if you want to use a 2.1 model for now we're just keeping the control weighted one the starting control stupid 0 and ending control Step at 1. so without making any real changes here I'm just going to type in woman I'm using the deliberate model I'm having no Styles here I'm going to generate four Images now we got four Images here and three of them the map is similar to the original we have some messed up teeth it doesn't really matter but this one here a lot of people have been complaining about the the face pre-processor and has been getting a lot of this now that I'm making a tutorial on this you might say wow you're a terrible YouTuber sub I don't want to learn to do this I want to make it work I got a couple of tricks for that let me just grab the seed we're pressing a little reuse seed here so whenever we generate now we're going to get the same results but we're going to change something here first thing that you can do to fix something like this if you encounter it is you can add something like my negative Styles those can be found in the description below so if I generate again now not only am I getting wildly better images all of them are well correct even the teeth are much better in all of them as a first render I would say they look pretty good now if you don't want to mess with my Styles or any other styles your own Styles or whatever you don't actually have to either there's another way to fix this is if you remove that you can actually prompt it so you had a woman with an open mouth in control nuts but you didn't get it in the output then you could prompt it in like say woman shouting now we're still using the same seed here now but hopefully we should now get four images with women with their mouth open like in the image here because we've told the AI the woman is in fact shouting so now it has like two inputs it's a controller then puts where it says an open mouth and the text input where it says okay she's showering so all four Images here are much better than what we had originally now if you combine the woman showering here with for example a default negative we're gonna have the digital oil painting and perhaps even a skin enhancer here and then we're just gonna wait this up a little bit so now if we run this again four Images where all the women are shouting the mouth is open similar to the control net we have the same pose everything and we're getting images that are more in line of what you would want or expect from a model like it deliberate so this is more similar to well the actual input because because I used these styles to create that input and as you can see the face are all having the same pose the head is leaning in a similar Direction and all of them the shoulders are kinda lined up it's a little higher here on the right side and and all of them so it's keeping the control net pose and the face pose exactly as intended so that's a super good super great and that's exactly what we wanted now if you would change this to the open pose face only model like we talked about earlier removing everything that has to do with the body so now stable Fusion can create the body well basically anywhere it's gonna create it obviously under here because that's where the body is compared to a face however we're going to see a lot of changes in the shoulders compared to our previous images where the shoulders were a little higher on the right side now we can see here it's actually the other way around now the shoulders are high on the left side and actually all our images because it is now adapted to the face and the head position and just looking at these from a first render this is a low res 512 by 5 12. this is before any imaged image upscaling or impainting has been done so we're getting some great results at this early step now you could continue in making the these even better I recommend you looking at one of my full workflow videos on that I'm going to link one of them up in the right corner right now if you are on a computer if not you can just find it in my video description I have now reset the seat to be random again and we're generating four new images and let's see if we can find one where the shoulders or the body is changing and it seems like in this third one here and the fourth one we're actually getting that so as you can see here the face are all lined up to the control net pose but now the rest of the Torso and the shoulders are because that is not controlled by control it anymore that is now randomly generated so here is you can clearly see here the shoulder the left shoulder is up here the left shoulder is up but she's more squared towards the camera she's at an angle and here the right shoulder is up again so this is a really cool way to control the faces and this super powerful one at that now let's say that you're looking for something like this but you want some variations into it and then I would change the ending control step here here it says we're going to start control Step at zero and then we're gonna run it for 14 of the full render and that's what's happened here you can see on the right one here on the right one here we're very close to our original a stable Fusion generation is random this one has started like the right one run for 14 and then it decided from this sort of output well actually the control that we had previously and then continue to recreate that into this so it's skewing what we had and it's the same in this one there's a slight angle in these three if we just look at them at a glance you would say oh they're all similar but they're actually angled much more different than the previous examples where we had control net from 0 to 100 so that's a cool way of working with controlling it and giving it a base to work with and that could be a great way for example to let's say you want to do a troll man shouting and if you would set that let's say let's even put it as a rough troll man shouting and if you would render this 0 to 1 which is would be a hundred percent of the render you would get some something that's again following the style of the input these are angle all very similar and you can clearly see that they have sort of the same base now they're not 100 similar or the same but very close now let's reuse that seed and again let's change this back to well maybe 0.1 so we'll only run for ten percent we might not even see the same pose in the end so in all the examples here we have the same sorta angle you can see that it's again similar but if you look closely especially on the top right one here or is angle a lot to the right and this one bottom left here with much more angled towards the camera so again a very powerful way to give a little chaos or a little Randomness back into your generation perhaps you have an input that isn't perfect but just similar to the composition that you're looking for now if you have a full character let's say we have this man walking here you're gonna want to be using open pose full if you want to use this particular type of face pulsing now in theory you can still use the Kanye or the depth map etc for faces also but I've found that the open pose is particularly powerful especially when you want to change the style we don't have to remake this into a man walking it could be anything we're going to enable we're gonna change into the open pose model and you can see here in our preview we have the face we have the body we even have the hands down here so it actually does hands fairly well now it's a little small and far away but it um when the hands are a little bigger and up you know like this it works pretty okay so let's have astronaut on the moon I'm gonna put our default negative the digital painting in and we're running this for four batches now I might have made a mistake here riding astronaut because in most of our images we can't see the face at all but in this one here and this one here we can actually see that is leaning towards the right obviously this isn't a good face but that's not what we're looking for at this stage now when the character is far out like this faces tend to be screwed up a bit by stable Fusion let me change this to something else so we don't have to mess with the device or the helmet here we're gonna do woman Viking Warrior instead and again as you can see in the images the pose is well pretty good in all of them but the faces are messed up we have four messed up faces so let's take the let's take the third one here send that between painting we're gonna paint on the face here we're gonna say Woman face make sure that you have masked content or Ridge original which is basically saying that we're gonna take what's below here and use that let the noise would fill with new material so keep original and make sure this is only masked it's usually set the whole picture so click in on the mask not about 0.60 noising more if you're looking for more changes and as you can see here in our results we have four new faces that aren't messed up anymore now you can do this even better by sending the original image into image to image upscaling and then in painting but again check my workflow tutorials on that so this was a quick look at the different options that are available for open post phase for faces in the control net 1.1 models now there is another one available from media pipe which is Media pipe face here and this is a separate one and it's actually I believe it's from a different I'm not sure if I want to say company but the different developers at least then you're gonna load the media pipe face here I'm going to link this in the description below now I do think the original works just fine but this has some strengths but the other one doesn't you can see clearly here that instead of all the dots that we were getting before now now we have a little more detail around the eyes the eyebrows and the mouth here so let's go back and take our woman shouting and quickly looking at these at a glance they're fairly similar to what we had in the control net 1.1 face models but you're gonna have to test for yourself see what works best for your use case like I said it adds a little more details pushed around the eyes and and the eyebrows here so at least now you have options to choose from and that's great because if one thing doesn't work out for you you can try the other and and vice versa I hope this little explanation helps you like And subscribe if you wanna but I'm not your boss so do whatever you want thanks for watching as always have a good one see ya