Differential Diffusion - Inpainting on Steroids!

Key Takeaways

differential diffusion is in painting on steroids it's time to work this method has two huge benefits the first is that it understands better what is going on in the image so it replaces what you want to inpaint in a more natural way and the other one you can see here is that you can use different gray values to Define where in the image you want to Place how much so you can do in painting basically on a pixel by pixel level so the darker the value goes in your mask the more the image is going to be replaced and you can see here some beautiful examples of that now this is not yet working inside of automatic 1111 however you can use it inside of blood diffusion I couldn't get it to run myself but you can try to make it happen now I have a reinstall of flut diffusion so this is completely fresh install and when you go down here where you find your scripts you can see here differential diffusion is one of the choices with their new update now down here you can enable it you can mask invert you can set the mask strength and you can also set a model here when you select these models these are going to download on the first rendering process so this is going to be slower and then and also down here you can load a mask now apparently if you don't select a mask image down here this is outo generating one for you but I got a null error so instead I'm going to show you this inside of comi where again I think this is a lot more powerful and also as always my Patron supporters get this workflow so you can try that for yourself I built here at the beginning a very simple classic process where you load the model you have your positive and negative prompt the K sampler and then the vae decode to generate the image so far everything is super basic of course we also have here our empty latent image now next we are painting the map here we have something that is called a preview bridge this is from the impact pack and is very useful because this will show you the image you can paint the mask and then you can process this further in your workfl to paint the mask in here you want to right click and then down here select open and mask editor when you open that you can then paint onto the image so here you can see the original image that I have and then you paint your mask in on the area you want to replace or you want to change and then you click here on save to note I have here an extra note that is converting the mass to an image just so you can see what is going on and I have C an extra note with a gausian blur for the mask this makes the mask soft on the outside and that blends it a lot better with the rest of the image now in that process to give a fair comparison I created a c generator that is sending the same seat to both of these Cas Samplers in the green area we have it with differential diffusion and in the red area we have it without which is the classic in painting just with a mask and as you can see here with the differential diffusion it understands the structure of the face a lot better and creates nice sunglasses on the face even the rim is nicely going into the hair of the woman this is by the way happening with the D noise of one which basically means it's completely generating that area new but as you can see everything is fitting to the face in a correct and nice way now when we look at the same image that is generated with classic in painting and you can also see that we are using D noise one in that case the result is not as good so the glasses are fading out here they're not going into the hair the eyebrows are not looking good at this point you might wonder where is the actual differential diffusion happening in here well um it is this little node up here that simply sits in between your model and the cas sampler it is small but it works well that's what the ladies tell me anyways and of course as always to install everything you want to go to this window here you want to click on the manager you want to click on update all to have the newest version and if you have any red notes inside of the workflow when you load it you want to click here on install missing custom notes and then here in that list that pops up on the right side you have the install buttons install everything that is suggested and then restart com for youi now you might think this is probably happening because the mask is too small and this is fading out because of it so I made the mask a lot bigger and rendered everything again and even though in the new result you can see that the glasses are there a lot more they are still not perfectly because here on the edge it's kind of strange how it's rendering it the eyebrows look a little bit strange the hair here is kind of missing and the glasses a bending over here where they are supposed to go into the hair now you might say this is probably because there is a blur around the mask and the value of the D noise is too high so I turned the value of the D noise lower and I turned of the Mask blurring but still on the right side we can see here the classic in painting we still have a problem here with the hair we have here a cut for the lip which is not good and overall the image doesn't look satis fying while on the left side with the blurry mask with the D noise on one we get a very satisfying result of the in painting on the first try let me know in the comments what you think about this new method I'm really happy with the results that I'm getting thanks for watching this video and see you soon bye oh you're still here so uh This is the End screen there's other stuff you can watch like this or that's really cool and yeah I hope I see you soon uh leave a like if you haven't yet and well um yeah