ControlNET: NEW Face Tracking Method - BETTER Faces than ever before!

Key Takeaways

better face expressions in control net there's another method for control net of course there is hello my friends how are you doing so this one is about the face rotation but also the face expression render this I want to see that in my Facebook group or in my Discord also check out my live stream tomorrow we will test this model play around with comfy UI and review your works okay let's talk about some facts here this new method is an early development that is only for stable diffusion 2.1 but I'm going to show you two models that work really well with that so let's get started here for the install process so for this what you want to do is to go into your automatic 1111 install folder look for the extensions folder and then here you have the SD web UI control net now I would suggest that you move this folder into another folder as a backup or simply delete that folder because we will replace it with a new repo that means a new copy of that control net that will be downloaded and will include the method for the face lion after we have deleted this folder here you click up here in the address bar and you write CMD hit enter this will open up your command window in here you will simply copy paste this command here I will have that as a link below my video so you can copy past it right in here hit enter this will download everything you need for this new method and install a completely new SD web UI control net folder with all the methods in there so you can still use all the other methods like Kenny depth map everything you used before but those are of course four stable effusion 1.5 only the phase method is for stably Fusion 2.1 as a very important difference after you have done this you want to go to this hugging face page again I've linked this Below download the control net SD 2.1 lie on face version 2 prune safe tensor file so click here on the download arrow and you want to save that in into your automatic 1111 folder in there into the models folder in there into the control net folder where all of your other models are now before we're going to restart our automatic 1111 we need two models that work with 2.1 the one model that I want to suggest to you is called Mishi anime this of course is for anime images but it works very well and has very beautiful results so you can download that here you download that into the automatic 1111 models folder and in there into the stable diffusion folder where all of your other models are the second model I want to suggest to you is called realism engine this can create portrait images can also create scenes like these Street Scenes here now in the description it says it is not advised for portraits but I tried it out it works pretty well for portraits especially when using the prompts from the realistic Vision model so you can go to that site here and just copy these prompts over work pretty well now again for the realism engine you want to download this again into the same folder so like I said automatic 1111 models folder stable diffusion follow now we have to do another step because this is 2.1 so you want to go to your web UI user.bet right click show more options and then select edit in set command line arcs you want to write minus minus no minus half and then save that file after all of that is done double click on your web UI user.bat to open up stable diffusion now first of all you should be able to see control net down here I have two control Nets because of my settings you probably only have one in here you want to check under the preprocessors that you have one that is reading media pipe underscore face and then over here for the control net models you want to have the control at sd21 lie on face version 2 prune to model in here so that everything is set up correctly if you don't see that model in here you click here on this blue button and this should reload all the models from your folder now you should be able to see that next you want to go to settings and when you're in here scroll down on the left side and click on control net app on the top we have a config file for the control net models this is a yaml file for you this says v15 but of course right now we need v21 so for a stable diffusion 2.1 you want to change that and then click apply settings this is very important but you also have to keep in mind that you change that back when you want to use the other methods again for now that is a little bit unconvenient until this method is rendered for stable diffusion 1.5 which I was told should happen rather soon after this has all been set up load here one of the 2.1 models I'm loading realism engine here write your prom the negative prompt in this case it is suggested to use ddim for that specific Model 30 steps for the sampling of course a resolution of 768 by 768 and then down here you want to load your photo to see that the pre-processing is working you can scroll down here and you can click on preview annotator results this will render this mask for you that should look similar to The Face on the left side after you're ready just click on generate and let it render and see what kind of results you're getting from that now here's another thing that I want to point out I have here two control Nets and you would think that you can use another control model in the second control model here but you can't because the second or all the other models that are in here for example the Kenny model are 1.5 that means when you use both of them together this will not be a applied to the image a workaround for that could be to download the control net models for 2.1 I have not tested that but that might be able to work so you can use multiple models at the same time here on the screen you can see some of the results I got write in the comments what you think about this method and please post your results in my Facebook and Discord group see you tomorrow in my live stream bye oh you're still here so uh This is the End screen there's other stuff you can watch like this or that's really cool and yeah I hope I see you soon leave a like if you haven't yet and well um yeah