Upscaling Images in Python with ESRGANs

Key Takeaways

what is going on guys welcome back in this video today we're going to learn how to upscale images using python so how to take an image with low resolution and increase the resolution by a factor of four so let us get right into [Music] ited all right so we're going to learn how to upscale images in Python today for this I've already prepared an image I want to work with an image of an old guy that I found on pixabay so it's copyright free and when we zoom in you can see that we already see some pixels um when we don't have the smoothing effect here you can see that the resolution is not very high I can rightclick the image go to properties image and you can see it has 640 * 427 pixels so we're going to take this image and get four times the resolution we're going to upscale it by a factor of four and we're going to do that using this model here so here we have the P torch model weights uh from the real ESR Gan model this is a generative adversarial Network again but it's an enhanced super resolution G so ESR G um and it's made for upscaling so you can get these model weights from GitHub you will find a link to this page in the description down below just go to the releases page of real ESR Gan and then you can just look for um X4 and then plus. pth and then you're going to find this link here get upom and then all of this real ESR again X4 plus pth so this is times 4 a factor of for upscaling we're going to use this model here these model weights so I'm going to close this now I'm going to move to my coding directory and here I already have these two files so the model weights in the image and now we're going to start by installing the packages for this video today and then we're going to code the simple procedure now I am going to make a couple of comments U regarding the architecture and the functionality but this video is not going to be to learn about the architecture this is going to be just application so we're going to load the model we're going to apply it we're going to set some parameters but we're not going to discuss the methodology in detail here so we're just going to use it this is going to be focused on the application not on understanding the architecture so what we're going to do first is we're going to install a couple of packages we're going to install basic Sr for basic super resolution we're going to install real ESR game and we're going to install pillow we're going to install numpy and torch so these are the packages that we're going to need for this video today I already have them on my system so I'm not going to install them and once you have them just open up a new python file and we can start with the Imports we're going to import torch we're going to import numpy SNP we're going to import from pillow image and we're going to import from from real or actually let's start with the basic Sr first from basic Sr architectur so Arts we're going to import uh from RR dbet which stands for residual in residual dense block network uh basically the idea is you have dense blocks with a lot of residual connections as far as I understand it but I'm not an expert on this architecture um import RR DB net and then also from real ESR Gan import real ESR ganner so this is basically here we have the model itself we could say and here we have a framework that uses it that uses the model so what we're going to do now is we're going to load the model we're going to load the parameters from the model from the pytorch file that we have and then we're going to just create a model inste iate it with parameters and then we're going to use um this framework here to create an upsampler and then we're going to just apply it that's what we're going to do so the model path is going to be our um our real ESR pth file and then we're going to say that the state dictionary so what we want to load into pytorch is going to be torch load model path and map load location is going to be torch device um I think we can use the CPU here because it shouldn't be too much you can also use your GPU if you want to uh just I think Cuda should work here not sure about this but I think just torch device Cuda should work uh we're going to get a couple of warnings here because of some package version conflict so what I'm going to do here is I'm going to say import warnings this is of course not a professional fix this is just trying to make the video not so distracting so I'm going to say warnings do filter warnings and then ignore so we're just going to ignore all the warning messages so that we have a clean output or no output at all so we're going to load uh this and in particular we want to load the parameters and we want to smoothen them with an exponential moving average or we want to take the parameters that have a smooth exponential moving average applied to them so we get params EMA and then we take the state dictionary and we say model is equal to R rdbn net uh we're going to instantiate this model with the following parameters we're going to say we want to have three input channels because we have RGB red green blue we want to have three output channels because the output is also RGB so we have an image as an input and we have an image as an output so three and three we want to say that the feature maps of the first um of the first layer are going to be 64 we want to have 64 feature Maps we want to have 23 of these oh sorry Number Block 23 of these uh residual in residual out uh dense block things so we want to have 30 uh 23 such blocks um and then we also want to specify the number of growth channels this basically means how many more feature Maps we're going to add um in every layer so we want to add 32 per layer and we want to define the scale this is the most important one to be four we want to have an upscaling factor of four and then we say model. loadstate dictionary uh State dict and then strict is equal to true so this loads the model parameters from the pth file into our model here and with that we can go ahead and create an upsampler using the framework so upsampler is going to be equal to real ESR G ER and we're going to say of course the scale is four come on what kind of autoc completion is this uh model path is equal to model path why do I get scale all the time I need to fix this my NE ofm config is not ready to use yet so model is going to be equal to model no I don't want to use scale thank you um tiling is going to be disabled the idea of tiling is you can have very large images that you split into tiles and then you process them individually we're going to not do that because our image is not very um very large then we're going to say pre padding is also zero we don't want to do any padding before we do the processing and we're going to use half Precision floating Point numbers just to save some uh memory and also increase uh the the performance or to speed up the computation so that is our up sampler and now all we have to do is load the image by saying image is equal to image open image JPEG in our case we convert this to RGB red green blue and then we turn this into an numpy array by saying that it's going to be NP array of image and then we say the output is going to be we're going to have an empty placeholder here for the return value the output is going to be upsampler do enhance the image and we're going to say outscale one more time is equal to four and then the output image is going to be equal to image so pillow image from array because the result is also going to be an ire array from array output and then output image is going to be saved onto the disk as output. jpack or actually let's go with PNG all right so let's save this and let's run this now if I didn't make any mistakes this should work take some time and hopefully we then see the result let me open up Nautilus here uh actually now this opens up Nautilus yeah okay let's just switch to this um so when I open the image the original one you can see it has some pixels here when I open the output image you can see that the resolution is much better I have to zoom much more into the image to get the same uh pixel effect so I have roughly the same Zoom right now maybe like this this has much better resolution now you can see it looks a little bit cartoonish so it's not going to look perfect but this is way better than the upscaling that we had like uh before models like this now just model is from 2021 so it's not super new but 10 years ago upscaling was very very bad and this is pretty pretty decent so the guy doesn't look very different I would say I mean a little bit maybe a little bit more cartoonish but it is the same guy but with better resolution so yeah this is how you can do simple image upscaling in Python so that's it for today's video I hope you enjoyed it and I hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye