Using SDXL Controlnet with Automatic1111

Key Takeaways

hey guys control nets for stability Fusion sdxl is now available for automatic 1111 and I want to show you how to install and use it for those of you who are not familiar with stable diffusion stxl it's a much larger model than stable diffusion 1.5 or 2.0 2.1 and it produces more realistic images with a much better phase generation it creates more sophisticated images with shorter prompts and it even can produce legible text also it's much better for larger images as the standard image size is now 124 by 124 pixels compared to 512 by 512 for stable diffusion 1.5 or 768 by 768 for stable diffusion 2.x I've been working with sdxl for automatic 11 11 for some time and it produces great results the only thing I've been missing using so far was controlling it which hasn't been available for automatic 1111. until now so let's get right into it and let me show you how to set it up and use it first of all you must ensure that you have installed the latest versions of automatic 1111 and control net so when you're using Windows let's open a new command prompt and change directory to the location where your stable diffusion installation is located then type git pull in order to install the latest version of automatic 1111 if you haven't installed automatic 1111 yet I will leave a link down in the description how you can easily install it now let's fire up the web UI hyphen user.pad and wait until it's loaded in the stable diffusion web UI go to extensions and if the SD web UI control net extension is already installed click on check for updates and then apply and restart UI to ensure that the latest version of control net is installed if you cannot find control it in your extensions list I will leave a link down below how to install it now let's have a look at control net at the text to image Tab and when you see something like control net 1.1.4 or 7 or later then everything should be fine next I want to show you where to get the sdxl base models as well as the control net models for sdxl so let's open a new browser tab and head over to civitai.com you could also download them from hugging face but I think that cvdi is more convenient once you are in cvti search for sdxl and then select the sdxl base model called sdxl now first download the V 1.0 VA effects which might take a while because the file is rather big and once it's done also download the refiner vae fix which will also take you a few minutes depending on the internet connection once your downloads are finished open your file explorer and move the models into the following folder stable diffusion web UI models stable diffusion now back to civit AI let's search for control net cnxl and when you open it you can see that there are tons of control Nets available I've downloaded them all but of course I didn't have the time to test each one of them but I will give you some examples about the ones I tried and I can already tell you that they produce quite good results and I'm going to take a deeper dive into these models in the coming days once you've downloaded the models move them into your stable diffusion web UI extensions as the web UI control net models folder and that's basically all we need to do and you can use them right away now back to our stable Fusion web UI and in the stable diffusion checkpoint drop down you can find the new models and if you don't see them just hit the refresh icon next to the drop down there's one more extension I would highly recommend you to install when you're using sdxl so in the extension step click on available and load from then search for XL install the style selector for sdxl 1.0 which will allow you to apply different styles when using sxtl I will show you later what I mean then again check for updates and apply and restart UI and we are good to go once the UI has restarted select the sdxl underscore V10 V A efix as your checkpoint file and I've already prepared a prompt to test it down below you can see the sdxl Styles region that's the extension we installed before and you can choose among a great variety of styles to apply to your sdxl renderings you can also see a refine a button so let's click to open it and from the refine at checkpoint drop down you can select the sdxl underscore V1 0 refiner vae fix and when you render your image there will be a second render bus with the refiner model which will add more details to the image and gives it an overall better look now let's hit generate and see what we get looks quite good just what we expect from sdxl please also note that the width and height of the image are set to 1024 by 1024 and that any lower resolution and sometimes cause quite unexpected results so it's no problem to use higher values for the width and height but don't get lower than say 768 by 768. okay now let's activate our new control Nets let's enable the first one and choose a guiding image I'm using canny as a preprocessor and I'm using one of the control net models which has downloaded from the models drop down just play with the different models and see what you like most I also click on controller this is more important but again just try it out and see what you like most for the second control net I use open pose but let's first choose the same guiding image then choose the open pose preprocessor and one of the models we just downloaded then again hit generate and you will see that the new control Nets are now working and they produce quite a good result I'm really excited that the new control net models for sdxl are working that well it's just what's been missing in automatic 1111 until now and I'm looking forward to using these models in the next future especially for video production which I hope will lead to much better results than in stable diffusion 1.5 I will keep you up to date and when I find out something new and meaningful I'm going to post another video but that's all for me today and if you have any further questions please don't hesitate asking me I do hope that this tutorial has been helpful for you so bye for today and see you next time