GPT-3 in Automatic 1111 - Pix2Pix Stable Diffusion Guide - Everything you need to know

Key Takeaways

create amazing AI images by using normal English language hello my friends how are you doing so today I'm going to show you a new extension for automatic 1111 how to use it how to install it and then at the end of the video a much simpler solution that has exactly the same functionality alright let's first get started here with the project page and you can first of all see here some pretty awesome examples for sunflowers by Van Gogh replaced as roses we add here some fireworks to this photo or you can add a cake instead of some fruit to this painting The Prompt is actually just normal English language no secrets no special techniques required the reason of course is because this model is trained with the language model jpt3 that you already know from chat jpd3 where it works absolutely amazing down here we have some more examples for example for example landscape where some ships have been added to the lake or the mountains have been replaced by skyscrapers now of course when you look closely here it's not just the mountains also the trees have changed but the result is still pretty amazing now the cool thing is on the top of the page there's a demo button here when you click on that this is loading the model on hugging face so you can try it out you can upload your own images add your instructions here down here you have a bunch of settings like for example the steps the seat the text CFG scale and the image CFG scale and then you can try out if this model is for you if you enjoy the results you are getting but of course let's look at how you can install this on your own computer as an extension in automatic 1111 and this is a very straightforward process so first of all go into your folder where you have installed automatic 1111 look for the web UI Miners user.pat and for that to right click show more options edit and here you want to add git pull before call Web UI because you want to have an updated version of your automatic 1111 install save that close that double click that and let everything run so it is installing the update that you need after this has finished simply load your web UI and click here on extensions there you want to click on available and there you want to click on load from and the first point here is instruct picks to picks this is exactly what we need so on the right side you have an install button click on that and in no time this will be installed as an extension to check if this is actually there you can click here on the install Tab and you see it here in the list to make sure that all of your extensions are updated you can also click on the check for updates button up here if you want to now here's another thing we we need and that is of course the model for that we need to go to this page here on hugging face there's two versions of the model one is the ckpt model and then we also have the save 10 Source model now the only difference between the two is that safe tensors is safer against being pickled but because this is an official release there shouldn't be much of a danger and the ckpd model can be used with more different web UI so that might be a benefit because the download is 7.7 gigabytes when you download this you want to download it into your automatic 1111 folder in there into the models folder and in there into the stable diffusion folder where all of your other models are when the download has finished close your window that you've used for updating and then simply start automatic 11 11 again after this has been finished and you reload your web UI you now have here the instruct takes two picks tab so click on that so you see this kind of interface and then you also want to go up here to the list and look for the instruct picks to picks model and load that from that list after that the process is pretty straightforward you have here you prompt your negative prompt here you can upload images but there is also a very cool extra function right below it because you can see here that it says batch Imports so down here you can activate this and you can set here a directory where you have multiple images so for example if you have 10 or 50 images of the Eiffel Tower and you want to add fireworks to all of them you can just link to that folder and then have all of them rendered through as a batch that's pretty amazing below this you can have the output patches this means how many versions of the images are generated at the same time you have the amount of steps 10 10 steps actually creates pretty amazing results but of course you can go with higher step counts if you want to you have here the seat you can set this to a fixed seat or random seat you have the text CFG scale this means how close it sticks to the text you have inputted the standard setting here is 7.5 I wouldn't suggest to go over 11 but you can of course try that for my experience the higher the Sea of G scale is the higher you want to have the step count so you don't get errors in the image the same for the image CFG scale now this defines how close this stays to the input image that you're loading because as the title of this extension says this is an image to image render below that you have the output image with so keep in mind that no matter how big the resolution is of your input images the output width will be defined by this value and in most cases will be lower resolution especially if you're working with photos which have a very high resolution of course stable diffusion and your GPU can't handle these high resolutions so let's get started here and choose some testing so first we have our Eiffel Tower I say add fireworks let's see what we get from this click on generate and there we go right away we have some beautiful fireworks in our image now here's the first thing I want to point out when we look at the original image you can see that the Eiffel tower has a very specific structure in the struts here and in the architecture of the tower now if you have these very fine details and this is also true for language in your image like text you can see in the output that these details are not there anymore they kind of look like it but they are changed in a lot of ways and this also will happen in a stronger way with text in your image so just keep that in mind but overall the output is pretty amazing it has actually put the fireworks behind our Tower but also if you look exactly at the image this has changed the lighting and the overall scene so that the color of the fireworks is reflected here on the ground with the same intense pink while the tower itself still has a very nice and warm orange because of course this is a different light source now let's go here with a different example we have this beautiful chocolate cake I'm getting hungry just looking at it now look at the composition of the cake all these drippings that run down on the side the Silver Plate this is on but also the stone wall in the background now when we look here at the result of our strawberry cake we also have here strawberry sauce running down you see the same Silver Plate down here and the same Stone background so a lot of the image parts aren't changed but the cake has changed to strawberry cake of course you will also see here that the strawberries are not exactly strawberries in some cases like this has turned into a strawberry snake so you might want to render the image several times with different seeds because the seed has a high impact here on the quality of the result now here's another example I'm going to show you and this is changing the hair color so here I said make her hair black this is our input image here you can see we have blue here we have these pink balls in the hair and other elements of decoration now the output image here looks pretty good it's not completely black there's still a little bit of blue left but the rest of the image is untouched and this looks pretty amazing now I want to point out here that this has also to do with the seeds so in some of the seeds these pink balls also have been turned into black however here is where we come to the limits of this model and its ability because again we have this image but now I want to change the hair to Blue and in this case all of the image is changing to Blue not just the hair and no matter how I change the input prompt what kind of text I was writing or negative prompt I was using this always changed all of the colors not just the hair and now at the end of the video I want to give you the big surprise that I promised you because this extension is exactly the same thing I showed you some days ago for playground AI for the photo or image editing so this is using the same technique you get exactly the same outputs with it and also on playground AI you can use normal English language but the good thing here is you don't need to install anything it doesn't use your Hardware so so if you don't want to jump through the Hoops of installing the extension and downloading the model you can actually just use playground Ai and generate up to a thousand images per day for free let me know what you think in the comments leave a like if you enjoyed this video and see you soon bye oh you're still here so uh This is the End screen there's other stuff you can watch like this or that's really cool and yeah I hope I see you soon uh leave a like if you haven't yet and well um yeah