UPSCALE any image for FREE with AI | Stable Diffusion

Key Takeaways

how I go from this to this for free with stable diffusion. Doesn't matter if it is an AI image or a real photo. I got to say that this is the method that I use for automatic 11-11 or Forge. So, I will also leave other options for Comfy UI in the description, though I don't use those as much. And with that out of the way, a mistake a lot of people make when they are new to AI is to generate large images directly from text to image, like 2048x 2048. This is most likely going to cause issues with the generation and not only take a long time, but also generate something deformed and nonsensical. That's why it's better to start with sizes that are supported by your checkpoint. Each one has its own preferences. For one, you can iterate faster that way and then you can upscale once you find an image that you like. If needed, upscale again from there. However, there are many ways to upscale. For example, if I take this image and just go to extras and upscale it by two or even by four, yes, the size of the image will be bigger and the edges will be sharp. But in no way, this is a good upscale. That's why the most common way to upscale from text to image is to just use high-res fix. Fixing the seed of the generation that you like and then just generating. If we compare the two with the same size, high-res fix has way more details and it looks way better in general. though the face looks weird in both. Still easy fix though. Just activate after detailer and generate again. There you go. This is the most basic form of upscaling that pretty much everyone who has worked with AI before uses. Simple, effective, and fast. And that works well for just playing around with AI and generating nice images. But sometimes you want to fully render a generation or even upscale a real photo. And this does not just mean having a bigger sized image, but also adding details. I will go over some of the methods that you have to do this from easy fast with decent results to hard or more work required with better results. But since I don't want you to get stuck in the parameters I'm using, let's first understand the basics. When we upscale, we are telling AI to invent new information that wasn't there before. To create this new information accurately, AI needs three things: pixels, freedom to change the image, and context. An easy way to add pixels is to just make the image larger. This doesn't mean AI will use them right yet, but we have them. The problem with making the image larger is that it gets harder to work with. Longer generation times and more BAM consumption. Also, AI doesn't like generating big images directly because it deforms stuff. So, if we go too big, we won't be able to work with it. But what about after detailer? That added detail without upscaling. Exactly. When AI generates an image, it doesn't mean that it is using every pixel to the best of its capabilities. So what after detailer does is use the same amount of pixels more efficiently by focusing on one area, regenerating it at a larger scale, and then pasting it back in. Of course, if the original image doesn't have enough pixels, it will not work. But it will use all of the original pixels that it can. This means that if we regenerate all the parts of the image in this way at an optimal size, we could get more detail for the whole image without making it bigger. But of course, it is not that simple. The freedom to change the image is most commonly referred as the noising strength and it is very tricky to get right. I will explain this quickly since most of you already know. But too high of a noising strength will change the image more than what we would like. And too little will make it so that we force AI to keep the original so much that it won't be able to add any details. We are looking for a sweet spot where the original image stays mostly the same but with new added details. An extra layer of difficulty is if the part you're trying to upscale is, for example, out of focus. If you try to upscale this, you might accidentally make it focused. And now when you look at the whole image, it looks kind of weird. In fact, one of the hardest parts of upscaling this way is keeping the cohesion on the overall image. Which is why one of the three things AI needs is context. Both visual context and prompt. What happens when we use after detailer is the same thing as using only masked in in painting where the focus area has limited context or padding pixels so that it can use the most amount of pixels to inpaint as all of this extra space is not going to be used. If we use whole picture we are basically giving AI all the context but only using the original selected pixels. So logically speaking, you would want all pixels to be used for the inpainting, right? For maximum resolution. But if I told you, okay, detail this, you have no clue what scale this is or even what it is. If I give you some little space, though, it becomes much easier. Meaning that we need to find yet another sweet spot where we have enough context while still generating at good quality. Trying to find the best quality with the least amount of context. Okay, now we know that we need to upscale the image until we have enough pixels and use these pixels as efficiently as we can. All of that while working under BR RAM constraints because let's say I want to use high-res fix and upscale up to 4K. Well, it just blows up and I have 24 GB of VRAM. Same happens if I go to image to image and try to upscale at X2 directly to 4K. And yes, I know I could probably optimize this and make this work, but not everyone has a 4090 and I don't think it is worth it for the little result it would bring. Anyways, so what do we do? Well, the most common and probably the best easy way to do this to this day is ultimate SD upscale. Here we are basically creating the image per part and then putting it back together. This makes the BAM consumption way smaller. You're going to pick at the noising strength and the size you want the output to be. The parameters depend on the model that you're using, but for most recent models, I use 1024x 1024. This method makes it very easy to see why the noising strength and context are a big issue when upscaling. 0.38 is a fairly low noising strength, but since AI lacked the visual context, I told it to make albido from Overlord with black wings. But this is a sky. AI is trying to do what I told it, but since it can't see that we already have albido in the image, it creates a new one. Even if I prompt for everything, I lack the ability to give every part of the image a specific prompt, at least with this method. If I don't want it to create messes like this, I need to give it more context padding or lower the denoising strength. Maybe both. If you can find the right parameters to where it upscales the image and it adds details, but at the same time it doesn't change the image too much or adds stuff where it shouldn't, then this could be a great option to use directly. A way to have more options than just the noising strength is using control net tile. This one here is for the illustrious checkpoint, but you can find a more general one for your model. Here you can play with the weight and ending step, allowing you to use a much higher the noising strength while retaining the original image. If this is the last upscale you want to do, I recommend fixing the image before upscaling since after this you will need more V-RAM. Another issue you might find with this method is the seams being visible. You can fine-tune the settings, but it will still be rough to keep everything coherent, like the leg here changing colors. With anime models, it will mainly make things sharper, but it won't be adding that much detail, though that's fine for now. But this method is really amazing when using flux, mainly with realistic images, but you can use it for other stuff, too. It's probably my favorite method with some added extra steps. So, here's how I upscale realistic images for the most part. First, I will move to forge so I can use flux. I will try this method with a realistic AI generated image for starters. Move it to image to image and play with the parameters a bit. I normally use the noising strength from 030 to 0.45 45 at least for the first pass. It can go lower if there are too many artifacts or weird stuff when generating. This is a play with it type of parameter. Then I go to scripts and use ultimate as the upscale with 4x ultrasharp, though you can use the one that you like the most. Here I'll put upscale at two times the original size and also put the tile size to 1024. Better for flux in my opinion. For mask blur and padding pixels, I'll also play with it, but not as much. Usually, the bigger the image, the higher I put them, but around 16 and 90 usually works for me. Seams fix is hit or miss, so I will first try without it, and if the seams are noticeable, I'll add it. Sometimes it helps, but most of the time not so much. Once it is done, you can see that this is like a little high-res fix. We have the original image, but with much better resolution. After this, I will upscale again pretty much with the same parameters, but with even lower the noising strength. Now, sometimes I lower the CFG scale as well, but I need to test that more to conclude if it's worth. This upscale takes some time, but we've gone to a pretty high image size. And if we zoom in now, you can see just how much detail there is in here and not that many problems. And just this could be enough for most upscales. But let's do this again. It will take a long time. And I'll lower the the noising strength a lot. I will also make the tiles even larger, like 1,512 to give AI some more context, but without losing the pixels on padding. Once this upscale is done, you can really zoom into it and see every strand of hair. Not without its problems, though. Again, with the skin being rubbery with too much stuff going on with it that can be adjusted with luras or prompting or whatever, but it's mostly something that you will have to work with in post. There are also some parts of the image that get weird context artifacts, too. And probably the most important one is that we start getting these noise patterns. I will show you how I deal with most of this in post later. But for now, even though I upscaled this image to 7K by 9K pixels or something like that, I have no intention on working with that big of an image. If you actually take this image and downscale it back to 4K or even to 2K, most of the details stay. I do this while comparing the image to the previous upscale to see how much it changed. Of course, you can adjust the parameters to get everything looking better, but I think this is decent. Just remember that the more you upscale, the more unnatural it looks since everything will be hyperfocused and have too much information. Again, I will go over some of the stuff I do to fix this later. First, let's talk about some of the extra steps I would use to improve an image like this after this type of upscaling has been used. We now have a lot more pixels than what we had before. This opens up to an after detailer type of upscaling where we only improve the details without making the image larger. So let's say I wanted to make her eyes have even more details. I can go to in painting, select the eye and generate. Very important to use only masked, which means I can generate a 1024x 1024. Again, this will be much faster because of that. And after the generation, you will see that the difference between one eye and the other is pretty big. The colors between eyes are now different because of the lack of context. I can just go and impaint both eyes at the same time. Though this will make it so that I use more pixels at once, sacrificing details to get better cohesion. Or I could play with the D noiseis. You do you. And yes, this means that you need to be careful with large masks. Because let's say I wanted to improve the quality of the ocean behind her. This mask is way larger than 1024x 1024. Even more so when in painting on the 9K image. So if I were to generate this, it would decrease the details and the resolution instead. Always try to keep this masks smaller than what you would think is the size you're generating at. I mainly use this to upscale little parts on the background or parts that I think are important since too much detail can take attention from the important parts. But for the change or earrings on degeneration, it works very well to make stuff make more sense. Yes, in painting one by one each little part takes a long time testing parameters, prompts, size, padding pixels, etc. But it can be worth when you stack all these details together over time. To help with that tedious work, I made an extension or I tried and with I mean chat GBD. The idea is that you can create the mask first, then select the parameters for each one of the masks. And once you have all of that, just hit generate and go it on the couch or whatever. The extension will loop through all of the masks and parameters, giving you a final image with all the inpainting done, even in batch if you need to. This extension has a few issues since I can't code. So, it takes a long time to do and I can't really test all the bugs myself. So I might share this extension now or maybe later after I post my next video when I feel it is working as intended. Right now I can use it but sometimes it crashes. There are some other issues with it. It took more than 1 month to do because basically I have to like tell Chad JPD what the issue is and all that stuff and it's a back and forth that's not fast at all. And by the way, yes, next video is about how to make extensions for stable diffusion with Chad JPT even if you don't know how to code. I also made one to upscale AI generated videos frame by frame and it works but again it has some issues. Back to upscaling. This method works nicely with actual photos as well even though it will of course give it a bit of an AI look. And for anime images, well it doesn't work as much. Maybe I need to test with newer anime based models for flux. For the background here, I like the skulls or the wings. It looks pretty nice. But for the face and body, not so much. I use Flux to upscale the background or to add texture to stuff like the hair or clothes. But then I also upscale regularly with either high-res fix or with ultimate SD upscale and control. It doesn't add that much fine details, but for anime it really doesn't matter as much. When I need extra details somewhere, I just use inpaint only masked. There are a lot of other methods for upscaling, but this is the one that I use the most and I probably like the most too. Ultimate does the upscale while choosing the right checkpoints and then deciding if control net is necessary or not. After that using inpaint only mask to one by one improve the details on the parts that I think are important. But that still leaves some if not a lot of issues that still have to be fixed if you care about the final result. So let's go over some ideas to do that. First for AI related issues like this super weird me. You should fix this before upscaling. I didn't because I'm lazy. These are AI generation related so I won't address them as much. What I will address are the upscale related issues. Okay, I will mix the best upscales together so that I have a detailed background and wings but maintaining the anime style for the woman. And yes, I've impainted some parts one by one. Here I realize that I messed up upscaling and the images don't match completely. But it's okay. Then lets me show you other issues you might have to deal with. Once I've mixed the parts how I want, I will zoom in and start cleaning stuff up. For starters, we need to know how to fix the small artifacts that appear all around, mainly in our albido image. There are a bunch of tools that make this easier. For small spots like these, you can use the band 8 tool. It is kind of context based and it works nice for little spots or lines. Just clicked over them and they should disappear. If you cover areas with defined lines, like the hair, try to follow the flow of the lines when clicking. This tool can help a lot with seams, too, by just clicking over them like this. But it won't solve all of them, and it will not solve all of your issues. A similar tool is just selecting stuff with your lasso and then clicking fill context fill. You can see that this tool works fairly similar to the other one, but allows for larger areas and a bit more control. You could also use generative fill if you have it or just in paint with stable diffusion. Moving on to other tools that you can use for the cleanup process. We have the clone stamp tool. This basically lets you click a part of the image and paint it somewhere else. With alt click, you can select what you want to copy and when you click that's where you copy it. Seems simple, but you need to get the hang of it. Knowing what to select and where to click. This ain't a complete Photoshop tutorial, so I won't explain it fully. But you can use this in zones where there is too much texture and it makes it hard to paint over, like the knee here. I can clean this mess up by just copying parts of the body with more normal skin and then just plaster it here until it looks decent. If you have a hard edge that you need to copy, you can go here on clone origin and just rotate the clone area to match what you want. And finally, just the brush. The brush is probably one of the best tools because it lets you do whatever you want, but you need to know what you want. The most common use for this is to just pick colors around the area you need to clean up with alt click and then paint over slightly by varying the opacity and the hardness to match your needs. There are lots of other tools out there, but with these, you should have more than enough to fix pretty much anything that you want. For example, I mentioned the rubbery skin on the woman when you upscale too many times, right? Let's try to fix that. I'll open my 4K image and import the 2K and the 8K1. I'll try to keep the details on the 8K image, but the natural feel of the 2K1 and with a 4K resolution. I'll just use adjustment layers to try to match the colors a bit better. Then I will try to lower the brightness of the highlights as well as diffusing them a little bit, adjusting the saturation and such until I like how it looks. You can play around with it as much as you want, focusing on what you want. I like trying to keep it natural, though I accomplish that as much as my current skills allow me to. But yeah, here you have a last before and after. And that's pretty much how I do upscales. Here's the image. And here's the zoom in. Let me know if you have any questions or ask on Discord. And thank you so much for watching. See it.