Stable Diffusion AI: 100 Cats Per Second…For Free!

Key Takeaways

great paper today fellow Scholars stable diffusion XL turbo why well because today we have these amazing computer games and simulations that run quickly and we measure this in frames per second then we have Offline simulations that run much slower in seconds per frame and today we have an AI technique that we can measure in cats per second you can create so many cats per second and it can do this too and surprisingly it may even help us train self driving cars will look into that and there's more you can also try this new tool right now and we will talk about this amazing technique too you can also try this for free too how cool is that so what was this cat thing this is stable diffusion XL turbo a supposedly quicker version of stable diffusion the popular open-source text to image AI the original version can do absolutely amazing things but it takes a bit about 20 to 60 seconds for an image and this depends on this setting the number of sampling steps we typically need 20 to 50 steps to create a high quality image the more steps the more computation we have to do and thus the longer we have to wait and here is an amazing new paper that promises what can that really be one to four sampling steps often in a single step that sounds incredible I mean if this was true we would be able to perform text to image in real time yes real time but wait a second this is not new creating an image in one to4 sampling steps has never never been a problem you can do it anytime you want with stable diffusion but unfortunately then you get this a blurry image no detail so why is this interesting dear fellow Scholars this is 2minute papers with Dr well it is interesting because this new paper allows you to create images quickly but also at the same time give you high quality images now let's have a look at the new technique wow that is as fast as you can type the results update almost immediately and no more blurry images wow so how quick is it well hold on to your papers fellow Scholars because it can create an image in 9 to 10 milliseconds yes there is 100 cats per second the resolution is 512 x 512 and the quality is is not bad at all it typically loses only against a slower version of itself ASD XL but SD XL has been surpassed by a new text to image technique yes we will have a look at that too in a moment so quality check mark But following your prompts closely is also super important and in that area check mark to excellent and there is so much more here it can also perform not only text to image but image to image translation one image goes in and it comes out transformed we have seen this in stable diffusion before and this helps you Unleash Your creativity like never before remember this earlier Nvidia paper where you could draw a landscape and it would almost immediately give you a nearly photorealistic image now it can do that too and not only with landscape images but with Apple's memojis too I love the quick iteration speed here and in fact people are already using it out there in the wild let's see how here you see an incredible example of realtime urban planning prototyping and visualization and you can even create animations with it all this for free and open source so how is this even possible it is possible through a technique called adversarial diffusion distillation luckily we have the paper with a detailed description of the phenomenon so here's how you do it first train a complex diffusion model this starts out from a noisy image then over time it learns to reorganize this noise into an image that depicts our text prompt but it does this slowly let's call it the teacher model now comes the magic we now create a a smaller student model that tries to mimic its teacher it learns how the teacher behaves and tries to reproduce Its Behavior but wait we already have the teacher model so why copy it well we are copying it with the student neural network so we retain the quality but at the same time this student network will be much cheaper and faster so more corgis and cats cheaper and and faster now hold on to your papers fellow Scholars because perhaps this could also be used to train self-driving course how well look at this cool new paper from Nvidia where they use real driving logs to analyze previous situations and even create new ones now here all of these agents are controlled by nvidia's Ai and are you thinking what I am thinking oh yes just imagine putting this in into an imageo image translator AI two more papers down the line and Bam you have a simulation where you can safely train your cars in challenging situations that actually happened or may happen imagine this in a similar manner to this earlier work where we can go from video game Graphics to real life and back but this time with real driving situations who and this new paper can tokenize trajectories mean meaning that it breaks down complex driving situations much like you would break down a sentence into words and then letters and it does it very very well how well look it is able to create more lifelike scenarios outperforming many many previous techniques it's like a mini video game with intelligent AI players what a time to be alive now as promised we are going to have a look at this new text to image AI that L at 1.1 billion images and learned to create incredibly high quality outputs for your tax proms you can try it here the link is available in the video description so how good is it well let's test it against stable diffusion XEL look at that approximately six to seven times out of 10 it is preferred over sdxl I think that is insane don't forget sdxl is a paper that came out approximately five months ago and it has already been surpassed Bravo and while we are looking at some of these eye popping beautiful images just imagine that two more papers down the line and I am sure that we are going to be looking at images and videos of these created in real time and all you need to provide is just a text prompt experiment tracking model evaluation and production monitoring for your deep learning projects and llm apps this is what weights and biases does and it is the best everyone is using it try it out now at wbme SLP papers or click the link in the description below