Faster Stable Diffusion Tensorflow-backend on Colab with Gradio Web UI

Key Takeaways

what's up welcome to one little coder today we're going to talk about stable diffusion tensorflow divyam Gupta who also created the one click installer for M1 Mac for stable diffusion recently came up with stable diffusion for tensorflow or stable diffusion that is powered by tensorflow what the VM has done is Divi must have ported the pre-trained models from pytos implementation to tensorflow and now it is available for us to use tensorflow how does it matter to you if you do not care about whether it is pi torch or tensorflow I was told um and I also read a couple of comments that because now it Powers it is powered by tensorflow there is some performance Improvement that it doesn't mean that tensorflow always has performance Improvement I'm not trying to make any statement please please do not drag me into any flame wash between white Arch vs tensorflow but the point here is that this is faster than what you would see typically on a stable diffusion demo on Google collab so that's one thing that for you to keep in mind so now what we are going to do in this video is us we are going to use the Google collab notebook shared by divyam and I'm going to take you through how you can use this python library or module a repository project stable efficient interflow to create your own stable diffusion images using Google collab also it comes with a graduate demo that the VMS kindly included so all credit goes to zvm I've just modified certain bits and pieces here and I'm going to explain you how to do this if you want to do this so if you have not subscribed to the Channel Please Subscribe 90 of subscribers 90 of viewers are not subscribers so subscribing to the channel would mean a lot to me let's get started this Google collab notebook will be in the YouTube description all you have to do is click it and then open it after you open it you would see an option saying save a copy in the drive that will make sure that you have the latest version of Google collab saved in your own Google Drive so that even if I make any changes you would not be affected by those changes the next thing that you need to check because you need to go to click runtime Click Change runtime and see if you have got CPU accelerated this requires GPU GPU will make inference or image generation faster so make sure you have got GPU now getting into the main part the first thing is this model is not available on pipei or radio so you have to upload it directly from git so right now I'm installing um a different version which is which has more performance Improvement but you can also install the version from Vivian so bottom line is just run this particular line of code that will install all the required libraries once that is done that takes a little bit of time once that is done now you can download the model and when you run this lineup Library uh sorry this line of code from stable diffusion underscore TF dot stable diffusion two things are imported get model text to image once that is done the first thing that you have to do is you have to download the model I've tried with a larger version of model even when the model is getting downloaded I'm not able to run this on Google collab because it hits out of um out of memory error so if you have got a different GPU you can try larger resolutions or larger size but right now I'm sticking to 512 by 512 so text encoder diffusion model decoded all these things are getting downloaded and you can even see from where this is getting downloaded at this point so you can see all these things getting downloaded the decoded texture encoder diffusion model and clip once all the required things are downloaded you have two ways how you can use this code the first way is a simple python usage simple python usage where you have to say that uh you need you need image from below that's just to display the image you can say text to image and then you can give the prompt like what I've done here a beautiful digital work of a dragon made from clouds by ISO Andrews and Peter murraybacker trading on Earth station image height 512 image with 512 text encoder text encoder diffusion model diffusion model decoder is equal to decoder and what's the batch size and you can just display the final image so if I run this I'm going to run this Live While I'm recording this video without editing so you get to see actually how long this one takes run this once you run this you can see that it would ideally take probably about 25 seconds um it might take more than 25 seconds it's it's not necessarily very strictly exact time that it shows maybe when when you get the video published you can see but it is good that it takes less than 30 seconds and then you are able to generate stable diffusion images so what are we trying to generate here uh digital artwork of a dragon that looks like it's made from clouds and let's see what is going to happen um it's more than 25 seconds I would say and here is the image I absolutely love this image it's always looks like you know this is dry this is cloud and this is dragon and it looks beautiful so all you need is just a couple of lines of python code to use stable diffusion this is right like quite the state of the art at this moment I mean there is I would say there is nothing like stable deviation in the open source at least at this point so you can just in bunch of lines of python code you can run stable diffusion on Google collab for free using tensorflow as the backend so this is quite amazing this is if you want to use the python code for example you want to integrate it with streamlit you want to integrate with plotly Dash you want to use it as a CLI you want to do something you can do this but on the other hand you want to use a grade your application and like I said this is a code Divya must shade to generate the greedy application grade your application gives you a web UI user interface where you can just go ahead and then give the prompt and then it will create the image that you want for this we need to import import numpy as NP import gradu as gr and first we need a function that function is going to be something that what we did here almost you need to take a prompt you need to take height and weight with sorry width and then certain things if you want more so what we are collecting here is input input image sorry image height image widths which can be pretty standard as well and then the number of steps so I might modify this uh to say that you know I don't want to collect input image height and width from the user I want to just give 512 by 512 I can do that so I'm sticking back to the same thing so height width number of steps and then you have the prompt input which is here so after you have all these things all you have to do is the same function the same thing text to image from print image height image width text encoder diffusion model decoder batch size and also the number of steps like the larger the steps and the better the image would be and then you take the image and then return it back to this so now this gradient does not use gradient interface rather it uses the latest API which is gradual blocks so now with radio blocks as demo first thing is you need to have the title this is the title right now it doesn't look like a title tag because we have used it without let me copy this we have used it without hashtag one which is H1 next thing is you are going to create a tab and what is a tab name the tab name is text to image for example tomorrow if you have got image to image I have got um in painting anything you can add that as more tabs that is graduate Tab and after that you are going to create a row and inside the row you are going to say I want prompt input I want image height image width number of steps everything is added inside as a text box with certain default values and then finally in the right hand side you are going to show inside the row you are going to also show the image output that is the image that has been returned from your the function and then finally I've got a button that says generate the image or you can call it anything generate an image and an image or you can keep generate and then finally you are saying that if the image button is clicked then call the function image prompt these are the inputs this is the output launch the gradier demo so I can just re-run it and then show you how long it takes for it to for it to run it would take a couple of seconds and you can see that uh this link would be active for a certain number of hours in this case it says 72 hours for this link to be active you need to keep your Google Cloud notebook active um that's a catch and you can see the stable division tensorflow demo you can see the prompt image height image which number of steps and I'm going to give the same prompt that I give a portrait of a sincere looking Indian girl girl oil and canvas something that I copied from lexicon.art so I'm going to just click generate and then you can see how many seconds it takes in the last time that I tried it took about 30 seconds let's see how long it takes it also depends upon the kind of GP that you've got uh I guess most likely I would have got it in Tesla T4 machine that if you are getting by luck anything else then it might differ if you want to run this on your own GPU um the steps are as same as what we saw the first thing that you need to do is you need to install the required libraries and then you just need to run this particular line oh this is amazing this is actually quite beautiful um and this is this is um this is developed by gradio sorry this is developed by stable diffusion for the problem that he gave now it says 28 seconds um it it might take a little bit more than that as well from my previous experience all you have to do is download run this and then this would work completely fine as far as I know on your local machine the only thing that you might have to do extra on your local mission is you might have to also install um you might also have to install tensorflow the first thing is blue tensorflow and then install this if you do not have it so this is another image and then you can literally go to lexicon.art Lexica dot part search for anything like for example this looks nice oops copy it come back here paste it run it and then see how it looks that's it um so you would have a working demo of stable diffusion powered by tensorflow running on your Google collab even with a web UI you powered by gradio and all these are completely for free so that's that's a good part about it so make sure that you give a shout out to divyam Gupta wherever you share this or uses give a start to the repository um give a shout out on Twitter and also share the pictures that you create like let's see what it turns out to be oh this this looks nice I don't know I don't know if it looks like Henry Cavill you know for for me somehow it looks like working Phoenix but whoever it is um it looks nice and uh that's that's the end of I this is still my favorite that's the end of this tutorial any questions let me know in the comment section otherwise I hope this tutorial was useful to you I'll see you in another Python tutorial