Feed-forward method | Neural Style Transfer #5

Key Takeaways

okay so this video is gonna be really nice and short I went ahead and reconstructed original Johnson's paper on fast neural style transfer since the CMM based approach as opposed to the Gatos approach which was optimization based let's see if some results I got using the for pre-trained models I trained beforehand so in the left column you can see the style images I used and on the right you can see the output from the models I trained so keep in mind that the the model in the top row is the only one that was fully trained so it's all like it did to e box of the whole ms cocoa dataset so that's around 83,000 images whereas the three bottom ones still need to be additionally trained now I'm just gonna do a quick time-lapse of the setup because we already did that in the previous coding video [Music] [Applause] [Music] once you have your con environment configured there are only two more steps to get some results so the first one is go ahead and run this resource download or script and it will just go ahead and download the pre trained models that I've uploaded right here to this your URL on Dropbox it will then unzip them and place them in this folder here Marvel's binaries so you'll have for pre-trained models on your disposal and the second step is just go ahead and run this sterilization script it's located here just run it with the default parameters and we'll get some results this is the result we get using the default content image that's located here just go to data cannon images and this one is used by default so I'm just gonna go and show you a little bit about how the sterilization script itself brooks i'm gonna go ahead and stop this one here mm-hmm oops and I'm just gonna show you where it's actually saved it saved to the output images directory here so you can be able you'll be able to find it later on in this directory okay let's see how the script works let me close this one and zoom this in a little bit like this so what we have here is a couple of default locations first one is for continum ages and we already saw that one that's in here we have output images where the images will get dumped binaries where we have our models and this is just basic error checking here basically checking if if we only have Pytor models inside of the of this binaries here directory and I just create the output directory where we'll dump images and here the arguments those are just the content image that you want to use that you want to pass through the model then we have the you wanna set the width of the output image and it's set to 500 by default here we have default model that's called mosaic and we just wrap all of those parameters called stylized static image function let's see what this function actually does so we go up side here we can first see we first want to check out if the device has GPU or only CPU support and then we basically just create this image path we pass it through this function called prepare image which will kind of add a badge I mention and do some normalization will check the function a bit later and I just instantiate here the transfer transformer model which is the actual model that performs the stylization we figure out the state so we just load the state from by torch model that's in binaries here we print some metadata and we finally load the weights inside of the the model and we just put it into this eval mode which is really important if you want to do inference I want to do this when I call this function here and finally we just wrap this into this context torch no gret which will basically forbid pi torch to calculate radians which will which would kind of be a huge memory overhead so we call the model on the can image we just paste it to CPU we convert it to non PI here and because we had the badge dimension that we previously added we gotta extract the zeroth bet here and that's the stylized image when we just save the image that's the image we saw a couple of minutes ago that was dumped here into your output image folder this one just check out that prepare image we saw in the utils here so this is how it looks like you basically just loads damage from the specified path specified by this variable image path and that will just basically create an umpire image normalized to zero to one range and then we'll pretty much add this normalization transform that byte order provides us with and we'll use image net mean and standard deviation to normalize our images after we apply the transformation here we just push to push the this this image onto the GPU if you have one and we just add this patch dimension in front because that's that's because models are always expecting dispatch dimensions so pretty pretty simple stuff in the next video I'll I'll cover training that they'll be a bit more challenging we're gonna figure out how to use tensor board and just debug and visualize our our training metrics so let me just go ahead and try one more image here so I'm gonna use image called figures and let me set the width here to 350 say we'll use the same model mosaic and if I start the one if I run it this is what we get so how good the output image will be actually depends also on the sides that you put here in the width so if I put 550 say here and I run that one it will have different stylization and it's much nicer if you ask me the next video will cover training until that time just go ahead and play with this repo try and figure out how the things work and I'll be a really nice here learning experience if you like the content go ahead and subscribe like and share the videos if you think they can bring value to somebody else see you next time [Music]