GANs with Fewer Labels

Key Takeaways

[Music] this video will present a paper from Google AI research on making generative adversarial networks work with fewer labels so for a quick recap Darren of adversarial networks taken random noise and then produce images such that they can generate novel data points this works by being trained against a another deep neural network a discriminator which classifies samples is real or fake and then this process iterates until eventually the generator can construct brand new images from the data set so the problem is that the state-of-the-art ganth require labels it's a supervised learning technique and this started with the conditional generic store networks where you took a one hot encoded class vector and embedded it with the generated image of the real image of the discriminator and then with the latent code on the generator so the key idea in the presentation in this video is that this technique is able to match the big Gann which is the state of the art on generating images on image net with just 10% of the labels and then it can outperform this technique using 20% of labels and this is super important because in the wild there is tons of data on things like Instagram Google Images and YouTube and it's really impossible to scale up labeling these data sets for the future of representation learning generation and computer vision so these this presents the overall results from the different techniques they test and this vertical bar indicates the state of the art using labels so we're gonna focus primarily on these s to gain and non techniques and discuss NS 3 in and discuss how this works so again these are some of the generated this is from the technique the S 3 again and this is from the began so again these are both 128 128 image net generate examples so the inspiration is self supervised gain and this is a paper that stabilized the training of generative adversarial networks by forcing the discriminating them 90 degrees 180 270 or not at all and then having a discriminative predict the rotation like how much was the image rotated and this technique is gaining enormous popularity it's known as self supervised learning and it's really like an unsupervised learning thing but it's sort of denoted this way because the task sort of lends itself to being modified in this way for a representation learning so in images it's done typically through rotations predicting the permutations of like a jigsaw puzzle by cropping out patches and then in language it's way more popular in language in language it's when you take like the some word in the sentence and then you predict the context so self supervised learning is really really popular in language models and it's becoming much more popular in image models as well so here's the idea you can get a combined self supervised learning with Ganz so one way of doing this might be to have a so you want to have your class label Y that you can embed in the discriminator and the generator and this will result in a much better again this has been shown that adding labels to Ganz really really improves them so one way of doing this might be to take the image and then train the self supervised model particular tations and then use these features to cluster the images into classes so use like k-means clustering to form like 50 auxilary classes based on the features learn from this self supervised learning task so another idea would be to even if you already have the Y labels is just to regularize the discriminators features with requiring the discriminator to also predict the rotation loss this is what is done in self supervised gans so here's another idea is semi-supervised learning so I'm sorry the terminology is so similar but semi-supervised learning isn't the same self supervised so much supervises this idea of you have some label data in your data set so let's say you have your 100% data samples you have ten percent are labeled so one way of using semi semi supervised learning with ganz is you have the discriminators features are used to predict the class label of the either real image or the image that comes from the generator so you would you you would train this classifier that is also using sharing these same features that classify the image is real or fake in the first place and use this to derive the late and then you would have a loss for the samples that are labeled so for the 10% of images that are labeled you would have some feedback for this part of the network so here's the idea of the of this paper you're going to combine the idea of predicting the class what the discriminator features and then you're also going to regularize the features by having it predict this rotation loss so the discriminator is regularized with the self supervision term it's still predicting real afaik and then it's going to predict the class label and then embed that class label into the conditioning layer so over all combining these three different loss function makes up this kind of thing this section here is the normal Gann part I mean yeah generally like you can see how it's a pretty complex thing but overall it it works out in this way so these are the results using different percentages of label data so it goes five percent temperature and the F ID met these are like two metrics of evaluating generated images on frequently used on imagenet and to compare Ganz in the literature so you see that it doesn't really look like it saturates too much and the F ID score it definitely saturates but the inception score seems to still have a bit of a big improvement it's almost five full points higher so again this is the bar chart that shows how the this technique improves as you add more labeled data to the semi-supervised learning component of it so remember they have the so supervised rotation prediction that is like normalizing the discriminator features and then you have this semi-supervised learning part which is used to predict the class conditional embedding in the discriminator model so these are some of the results of the model to showing the latent space interpolation works well and then this is when they scale it up for a 256 by 256 target output so some concluding thoughts is really interesting to see semi-supervised learning and ganz being merged together and it also would be really cool to see this technique that was presented her pretty recently as well where they normalize they have the data that's unlabeled and then they augment it using like rotations and whatnot then they try to make sure that these class predictions are similar to each other so it would be interesting to see you know add a fourth loss to this network as well so thanks for watching this video on making generative adversarial networks work with fewer late labels please describe to Henri AI allows for more deep learning videos