Stable diffusion 2.0 Released

Key Takeaways

Hello friends so great news the king is back stable Fusion 2.0 is out and that means no super stable effusion 2.0 no fake 2.0 the real official 2.0 of stable Fusion made by stability AI so let's check that out oh and if you don't care about stable Fusion 2.0 and just came here for a joke I got you so five ants rented an apartment with five other ants they're now 10 ants all right so we're here at stability ai's blog and we have here stable Fusion 2.0 release so this is great news everyone so we had the we had a 1.4 we had a 1.5 and now we made a jump to 2.0 and this is stimulated AI which uh or emad is the face of that one it is our pleasure to announce the open source release of stable Fusion version 2. the original stable Fusion version 1 led by compass change the nature of Open Source a models and spawn hundreds of other models Innovation all over the world it had one of the fastest climbs to 10 000 GitHub stars of any software rocketing through 33 000 stars in less than two months so that was just some history here's the team working on it we have stable effusion 2.0 delivers a number of big improvements and features versus the original version one release so let's dive in take a look at them yeah that's exactly what we're gonna do it's an example image so we have a new text image diffusion model and just right off the bat here what this means is you're not gonna get a new checkpoint file that you can just download from hugging face like what happened in 1.5 with a new diffusion model every user interface is going to make need to make some changes so at the point of recording this it's not possible to put this into automatic 11 11 or whatever user interface you use right now but when you're seeing this maybe because there's already been pull requests to update some of the most popular ones so give it a day or two and we're probably gonna be well updated this has here stable Fusion release includes robust text to image models trained using a brand new text encoder developed by a layout which with support from stability AI which greatly improves the quality of the generated images compared to earlier version 1 releases the text image models in this release can generate images with default resolutions of both 5 12 5 5 12 pixels and 768 by 768 so this is great news now you have a native higher resolution which means I mean of course you could do a high resolution previously but doing it natively means that the model has been trained or fine-tuned really on on images that are 768 plus seven by six on 768 by 768 you can start with higher resolution images and then move upwards from there these models are trained on aesthetic subset of the Leon 5B data set created by the deployed team adds Wai which is then for further filtered to remove adult content using Lions NSFW filter so this is good news for some and bad news for some you're gonna have a not safer work filter which is great for professional use and just general family friendliness for some there's going to be a limited Edition but I'm sure they're gonna be there's gonna be workarounds for people that need that I think generally for professional use which I use the AI for mostly this is actually a good feature because there's been some sort of like a limitation of what the what what you can't do especially in a professional environment and for example just like doing a YouTuber streaming you can't live render anything because anything can pop up I'm just gonna get like banned that's not great because here are some examples of images produced in the native 768 by 768 another new features super resolution upscaler diffusion models so stable diffusion 2.0 also includes an upscaler diffusion model that enhances the resolution of images by a factor of four and here's an example of that so the model is upscaling an image that is 128 by 128 and into the high resolution of 512 by 5 12. so they say here combined with the text to image models which can get the images up to like what we talked about earlier 768 and now we can generate images up to well not up to it can generate images by to 2048 and as I said or even higher I think this is a like a solid number until it starts you know losing a lot of detail this is this is kind of cool the depth to image diffusion model now the Forum has been working a little bit with depth mapping and minus but it hasn't been that widely used in just regular stable diffusion models previously but now they have implemented that and it's they call it depth to image and it's basically you can have a depth map which well this can be an example of a depth depth map and um then your text to image or depth to image Generations will base their results on that so as you can see all the results here are based upon this depth map so it's a think of it as image to image but more advanced really and it says your depth image can offer all sorts of new creative applications delivering Transformations that look radically different from either Ridge from the original but which still preserve the coherence and depth of that image you can see the white image here what that one that pops out that's the depth image and then it generates out from that so that's really cool and that's gonna help especially in professional use I mean let's say you're working with everyone says oh mid Journey V4 that's that's so great yeah but it's it's I mean yeah you can get great images but it's so very limited you need to be able to control the AI generation like not 100 but but close to it if you're going to use this in a professional environment the demands and specs are so specific that you just you just can't deliver okay here's the beautiful image oh it's done I mean it works with Facebook and Instagram and stuff like that and well some uses but most of the time you need to be super specific and stable Fusion is the king of that and has been has never been dethroned in that regard and updated in painting diffusion model now in painting was improved a little bit in 1.5 and well I hope to see much better Improvement here in 2.0 in painting has been well it's it's been improved but it has been one of the weaker aspects of stable diffusion so far and that's also taking account the professional use in painting is an extremely powerful tool let's say you're working with like a composition or something that you have previously it's like you have a folder from a photo shoot you can just click quickly in paint a little bit that's going to transform the whole business and again just like the first iteration of stable Fusion we worked hard to optimize the mode to run on a single GPU so as I've understood it it doesn't say so specifically but I've heard talks about lower vram requirements all around we'll see about that I haven't tested it thoroughly yet we wanted to make it accessible to as many people as possible from the very start we've already seen that when millions of people get their hands on these models they collectively create some truly amazing things this is the power of Open Source tapping the last potential of millions of talented people who might not have the resources to train a state-of-the-art model but you have the ability to do something incredible with one this new release along with this powerful new features like depth to image and high resolution of scaling capabilities will serve as the foundation of countless applications and enable an explosion of new creative potential well yeah I don't doubt that okay so for more details yeah here's the GitHub link I'm going to put all the links in the description below so so check that out but remember as of this recording you can't just download the checkpoint and start working but give it a few hours and um most tools I assume will be updated for this because this is huge news huge news and again this is the real official 2.0 now again if you are if you are like a programmer or no coding uh very well you can get this to work there's a information in GitHub how to manually start it up we're quickly gonna dive into the GitHub as well so here's stable Fusion again 2.0 here's some example images so this repo contains stable diffusion models trained from scratch and this is important they have been trained from scratch and hopefully with experience and knowledge about the previous ones so this is again a huge update and we'll be continuously updated with new checkpoints so you're gonna have the you're going to have the 512 and the new 512 by 5 fill model if you're gonna have the fine-tuned 768 by 768 model so you can choose which one you want to use some of the news here here we talked about the upscaling the new upscaling and the new depth depth map with saw some examples of that here's another one here's a text guided okay that was the end painting model yeah that was we saw that previously okay so basically here if you know what you're doing you can update an existing latent diffusion environment and you're gonna need to run calm down and install the diffusions here exformers again are available and you should use them it's going to lower your vram requirements by a lot and speed up the renders as well and then just need to run that compiling here so this is just a graph of the comparison of the different models so the new 2.0 is the blue line here I'm not gonna delve too much into that some text to image examples yeah you can check this out bears by yourself I'm not going to delve into this most of this was in the blog post already but again I'm gonna post links down below yeah so there you have it stable Fusion 2.0 and to summarize what you have is a brand new latent space trained on larger images so you're gonna get high resolution native 768 by 768 better compositions a new upscaler depth to image as a new feature the not safe for work filter all that good jazz so yeah thanks for tuning in I hope this will be available when you watch the video and if not well just wait a few hours and I'm sure it will be updated in most interfaces have a good one see ya