How to use Stable Diffusion v2.0 on Colab with Gradio UI using Diffusers

Key Takeaways

what's up coders welcome to one little coder in this table diffusion tutorial we're going to learn how to use stable diffusion 2.0 on Google collab using hugging face diffusers at the end of this video you will have you would have learned to build a radio application that looks like this also if you don't want to build this by yourself this Google collab notebook will be linked in the YouTube description all you have to do is click the Google collab notebook and then enjoy your creativity through stable division 2.0 if you are not familiar with stable division 2.0 I've already made an overview video which I'll link into the YouTube description make sure that you check it out let's quickly see what this new radio application or a very simple Radio application does so it takes a text prompt and then it uses the latest stable diffusion 2.0 model via the diffusers hugging face diffusers library and then it generates an image so I'm going to quickly go to lexicon art where we have got a lot of prompts I'm going to copy a prompt here and then I'm going to come back to this and then I'm going to paste it so ideally the image wouldn't look exactly the same because stable diffusion 2.0 uses a different text encoder so which means it's it's not the previous encoder so a lot of prompts that we're giving good results for you in the previous version would not be true anymore and also I've read a lot of comments that they stripped artists from their training data which means most of this might not be useful at all but just for this simple exercise we are going to leave that out and then we are just going to use this as a reference point for us to understand how to generate images using stable division 2.0 but just because we are seeing the image I can actually see there is a huge difference in the way images are generated maybe this is certain parameters that we are not giving right now but that's that's what we have got so let's let's see the image that we have got so this is the image that we have got and we have got a prompt and we can generate image so now let's get into the details of how to do this by yourself the first step is this Google collab notebook will be in the YouTube description but if you're starting from scratch create a new Google collab notebook click file and then click new notebook once you click new notebook then make sure that you go to runtime Click Change runtime and then select GPU so you need GPU for faster inference so make sure that you have got a GPU once you have got the GPU just make sure that you can test whether the GPU is available and the easiest way to test an Nvidia gpus the bank Nvidia SMI that will tell you what is the type of GPU that you have got we have got a Tesla T4 with 16 gig machine which is good the next thing is we need to install the required libraries the most important is diffusers that is going to let us access stable diffusion the next thing is Transformers which is a dependency accelerate is to speed up as the name suggests but again um accelerate as a lot of usages but if you do not use accelerate in this this notebook will run much slower than what it is ideally right now and then some other dependencies and finally gradu for the UI development GUI graphical user interface once you install all these libraries the next thing that you have to do is from diffusers import stable diffusion Pipeline and Euler discrete scheduler in this particular stable diffusion 2.0 we're going to use a different scheduler from what we have been using before so then the next step is Define the model ID which directly goes to the model link from hugging face model Hub so this is where the new stable division 2.0 is added you don't have to accept terms of services or you don't have to do any check mark which also means that you don't need to authenticate or you don't have to have hugging face token this is a very important step ahead because that was creating a lot of friction for a lot of my users and they were getting stuck in the strip rather than they can you know instead of enjoying their Creative Juice they were actually stuck in the loop of creating a hugging face account and then the token and then everything else but right now good thing is model is available even if you have do not have any token or doesn't matter the next thing is Define the scheduler and then create pipe if you are familiar with stable diffusion pipeline you need to create a pipe from pre-chain model ID scheduler revision and torch type and which is which is just a just the data type to make it easier for memory usage then finally move the model to the GPU device in this case code and this is going to download all the models which will take a couple of minutes and the next thing is um it says make sure xformers is installed that's strange because I thought I installed explorers but it couldn't figure out but there's a game you can fix it when you are doing it by yourself the next step is Define a prompt in this case I have said a realistic photo of 3D cartoon style of a cat in a space suit with Milky Way background so the next thing is after you define a prompt the next thing you need to do is create a function I'm calling the function text to image and the argument is a prompt and then inside that we are using pipe prompt and the images thankfully are 768 by 768 I'm not giving any guidance scale here but you can also specify the guidance skill guidance scale equals maybe 10 or 7 whatever you want and once you get the output previously what we used to do is dot images of 0 but right now I'm just leaving it here and then save the image then return the image now let us once the function is defined which is not happening because we have the greedy application running stop to create your application just it's in my demo so I'm going to run this once again and I'm going to call the function which is going to create the image and the prompt is something that we have already created here and uh it's going to create the image using the function text to image and as you can see this is happening real time so you know how much time it takes for it to generate one image which is a 768 by 768 image and in this particular case we are using the stable division 2.0 model that got just released two days back and this is the image it looks pretty bad but like I'm not here to teach you prompt Engineering in this video this video scope is purely how you can access stable efficient 2.0 on Google collab on a free GPU of course so now we have got a successful creation at the end so what we can do now is if you have not installed radio installed radio and just in two lines of python code you may not believe me but this is true just in two lines of python code we are going to create this beautiful interface which is import gradu as gr and then gr dot interface greedy has two main classes one is interface the other one is blocks blocks is for more advanced UI for a very simplistic and easy UI I'm sticking to interface the first one is a function which you want to process when this input is triggered and this output is generated so as you can see for our function the input is a text prompt and the output is an image and as usual you've got the function the input and the output and finally launch that share is equal to true so that you can get a URL debug is equal to true you can trace if there is any error and run this and you're going to get this web interface so what does it mean it means you don't have to go to the stable diffusion default let's say the spaces and then when you try something it's going to say the server is busy or something like that you are not bound by what is available but what you are doing right now is if you follow this tutorial you have this available for you just click submit then you are not on a queue you are not waiting anywhere all you are doing is thanks to Google for its free GPU you can use and generate images using stable division 2.0 once again the image that you right now see may not be the perfect image because we are not playing with the prompts in fact I have not got enough time to play with the prompts to understand what kind of prompts are doing good like I said at the start of the video the problems that were working fine on stable division version 1.5 or 1.4 would not necessarily work fine here because this uses a totally different text encoder the training data set has been changed a lot of artists have been removed which means it's it's what it is so but at the end of this now we have got this image let me quickly go grab one more prompt so this looks nice let's copy this and then go back to our application and then paste this then click submit this is a very simple prompt this is a highly detailed three-dimensional retro futuristic Android Android is like the Android made out of pasta going for a walk outside very creative but I'm not sure if the image that is going to be generated will be as same as what we saw before in the previous stable diffusion model so let's wait it took about 25 26 seconds oh this looks pretty ugly at this point but yeah please please make sure that you feel you change the prompt and then find out what is working fine but the overall idea here is that we have successfully learned how to use hugging phase diffusers to use the latest stable division 2.0 not just you use as a as a like a code but also have a very good user interface you can give the prompt and then click submit if you want to improve certain parameters you can increase the number of steps like for example you can go here and then see what kind of what kind of parameters you can use also you can also go to the GitHub repository and then see what are all the options that are available why do we have to use particular scheduler and all those kind of details but until that this should keep you busy to create your new stable diffusion images if you have got any question let me know in the comment section otherwise I hope you find this video helpful in learning how to use stable division 2.0 on Google collab for free GPU see you in the next video