Build an AI Artist - Machine Learning for Hackers #5

Key Takeaways

[Music] [Applause] only humans can make something this incredible hello world welcome to serology in today's episode we're going to talk about building an AI artist art is the embodiment of The Human Experience it's a synthesis of all of our emotions and experiences in life you might be asking yourself is it really possible to build something that can do this the answer is yes it's incredible we can train a neural net to learn an artist style until it to modify a picture into a painting in that style it all started when the Google research team released a blog post called Inception ISM they trained a deep convolutional neural network on a huge data set of images so that it could start to detect everyday things like dogs and buildings once it was able to do that they gave it a novel image and asked it to detect an object it had learned in the image if it saw something in the image that looked even slightly similar to what I had already learned say a cloud that kind of looked like a dog it would then modify the image to look more like a dog this resulted in some pretty crazy pictures then another group wrote a similar paper called neural algorithm for artistic style but they repainted an image in the style of another using famous paintings as the base image so when they trained their neural net on Star KN then gave it a novel image it would modify the image to look more like Star KN by artify all of its features since we've gone over convolutional neural Nets in a previous episode at a high level we're going to really deep dive into the code for this one and try to understand exactly how this process works we're going to go through the necessary code to recreate the results from the neural style paper in Python using the Deep learning library Kos let's get started we'll start off by defining our arguments when we run our script we Define the base image the style image and the output image we'll create variables for them then reference a pre computed weights file called vgg16 these are just a bunch of pre-computed synapse weights trained to recognize everyday images which will later add to our neural net as a starting point we'll also want to initialize booleans that define whether or not to rescale our image or maintain their aspect ratio based on user input then we initialize our variables for style and content weight so what do we mean by style and content weight well in the neural style paper they found that when they trained a neural net to recognize the painting style of it artist the learn features in the initial layers were style based like you know how when you train a neural net to recognize a dog initial layers detect edges and the next ones detect shapes and more complex shapes and a whole dog well those same layers of abstraction apply to paintings but what they found were the initial layers the edges and curvature and other low-l features equated with style they also found that the highest few layers were more based on content so in the starting night photo the higher levels would be that dope Sun thing and perhaps a collection of stars the lower levels would be the curvature of the night sky and the color scheme in this way CNN helps separate content from style and mirror the capabilities of our biological Vision we then need to Define how much we want to weigh one or the other because we can optimize for one of them or both depending on how much we weigh each we'll get a different output we'll set our image dimensions then create a tensor representation of our base image style image and output image then we'll combine all three into a single input tensor a tensor is a multi-dimensional array an example would be like colors textures each of those would be arrays as well so colors would be its own array containing the seven main colors then each of those colors will be an array of sub colors we convert our image to a single tensor because it's an easily parsable data structure for our neural network tensors help reduce the high dimensionality of our images and that in turn reduces the computational complexity now we're going to want to build our model which is going to be a convolutional neural network we'll add in our input tensor as the first layer then we'll start defining the other layers we're going to add in 31 layers to our neural net there are three types the convolution 2D layer means it has a set of learnable filters which have a small receptive field the receptive field is a subset of filters that are used to connect the neurons to a local region in the next layer instead of every single other neuron the zero padding layer helps us control the size of the output volume by padding zeros across the border then there's the average pooling layer pooling is a concept in CNN where we take the input image and split it into a set or pool of rectangles pooling helps us avoid overfitting and reduce the amount of parameters in computation by only using a subset of the image as a representation rather than all of it the idea is that once a feature has been found the exact location isn't as important as its rough relative location to other features we'll take the average value from the pool the activation function is called relu or rectified linear unit the reu function is faster than sigmoid without a huge difference in generalization accuracy so we'll use that the numbers here are the number of output filters and the length and width of the input image we'll name each of our layers as well for reference now that we have our model We'll add in the preu weights we called earlier then we're going to Define our loss function for the style content and total variation aka the uniformity of the image loss functions help us calculate the difference between the expected output and the actual output we're going to take these loss functions and combine them into a single scaler or number then we'll get the gradients of the generated image using the loss which helps us map the color scheme our last step is to train our model by minimizing the loss using back propagation the back propagation algorithm we'll use in this case is called limited memory bfgs lbfgs helps minimize our loss function and is space efficient in that it only stores a few key vectors instead of all of them minimiz ing the loss function means modifying the output image iteratively so it looks more and more similar to this artistic style we want scipi gives us a technique as a built-in function and after training is over we can rescale and save the output image that's about it this is what happened when I tried out the algorithm you could also apply this algorithm frame by frame to video content and with more data and computing power it's only going to get better I hope that in the next few years we'll be able to generate not just paintings but other forms of art like sculptures and interior design and facial hairstyles for more information check out the links down below and please subscribe for more ml videos for now I've got to go fix a compile time error so thanks for watching