Self-Supervised GANs

Key Takeaways

[Music] this video will explain a study on using self supervised learning with Ganz to stabilize training and prevent catastrophic forgetting and cyclic problems with training Ganz so the headline idea is that this paper utilizes the rotation self supervision task to rival supervised Ganz where you use the class labels on Ganz and obviously we want to get away from doing supervised learning with Ganz because scans are useful unsupervised learning technique for feature extraction and also envision in general we have access to enormous image datasets and we can't possibly label all the data so this technique is going to improve image generation and the representation quality of when you use this as a technique to Train classification models so rotation classification is is one of the most popular self supervised learning tasks in addition to things like relative location colonization and I'm like super resolution and these jigsaw puzzle kind of things but so the idea is basically you take an image and then you either don't rotate it at all or you rotate it 90 180 or 270 degrees and then you'd have a network predict how much the image has been rotated so they're going to integrate this part of the rotation task within the Gantt framework so Ganz already work where the generator produces a fake image and then a real image is sampled from the data set and then the discriminator is gonna tell real fake based on these two but now what it's going to do is the images they're going to go into this pipeline and they're going to be rotated so the discriminator is classifying them as real or fake but it's also predicting the rotation of the image so the original gann objective is this loss where you have the expectation that the sample is from the data set and that you predicted that correctly or that it's from the generator and you predicted that correctly so what we're gonna do to modify it with the self supervision task is in the generator loss you add the successive discriminator in predicting the rotation on the generated image the X from PG means X sampled from the generator and then you know the prediction of the rotation angle you know the rotation equals are given this image and then on the discriminator side you the losses the sample from the data set and then the loss on this so the discriminator isn't actually updated with how successfully it predicts the rotation on the generator so for this reason it's called collaborative adversarial learning because the generator has the discriminators rotation success in its moss turn so that it's in the best interest of the generator to produce images that the discriminator can successfully classify the rotation angle oh so this is an interesting idea that they generated and discriminator our adversarial with respect to the real fake tasks and then they also have this other thing that they're doing together in a way so the motivation for this is catastrophic forgetting and deep learning and this image here isn't from Ganz it's from a classifier so what happens is if they train the model on the task of classifying with one class from the CFR ten data set and then they switch the class and every time they do this and switch the classes you see this downward spike as the discriminate urban as the classifier no longer has the kinds of representations to classify these images so see here it is probably classifying like cats images of cats and then it goes to dogs or something so in Ganz we have cyclic training this is one of the most common failure modes in addition to like a mode collapse and the discriminator I like dominating the generator so imagine that the generator the discriminator sends a signal and the generator jumps from this kind of distribution to this one and then the discriminator fails at this one because it's completely forgotten about this so it jumps back to this and then it continues this cycle forever and that's because the discriminator as the generator starts producing like this B set the discriminator is completely forgotten about a and then it learns all about B so the generator goes to C and then some and then it goes back to a to fool the discriminator because the discriminator can't continually learn the changing distribution of the generated samples compared to the real data set so additional motivation is let's say that you've got the optimal discriminator at convergence where the generated data distribution is identical to the original data distribution the discriminator at this point just outputs hi-five regardless to the input because you know if you predict a generated sample it overly confident predictions will kill it so it'll just do 0.5 like I don't know I don't know every time so when this happens if you take these representations from the discriminator and use it for as features for a classification model they get really bad because the discriminator no longer has a sense of visual representation because it's just outputting 0.5 regardless of the input so these are the initial results of the self supervised gain and this modulation and the modification to batch normalization compared to conditional gans and then unconditional gains so we see already that the self supervised gain is outperforming the unconditional again which is just again without the rotation task and the conditional gannets the gain but with the class labels so now they're gonna scale it up they're going to use the big gain model which is an enormous general diversity of network model they're going to use an enormous amount of computational resources with 128 of the Google TP uv3 pods and then they're going to increase the batch size up to 2048 so using this they're also going to search for some hyper parameters on the gradient penalty and spectral normalization which control D like Lipschitz constant of the discriminator basically a function to make sure I'm in a limit like a normalization method to make sure the discriminator doesn't become too strong for the generator and then they also iterate on the hyper parameter of how many training steps should they discriminate or have per generator set because the discriminator is so heavily normalized with these gradient penalties and spectral mobilizations that sometimes you might want to run it for twice to catch up so these are the some of the samples found from their best model in the image net these are pretty convincing results given that there was no labels provided to the network and then this is the they use the F ID metric for you know presenting the quality of their generated samples and on this metric the lower score is a better score so this just shows like as it's training you know you sample some of it and then you calculate the score so you see on image net how the unconditional gain collapses and this is like a failure mode this would be a really high score and then this is the rotation quality showing that you know when you when you train the again and then you take the discriminator is visual representations and use that to train a classifier you see that when you use rotation only the self supervised task of this one you get about twenty twenty percent unconditional does horribly and then the conditional Gann doesn't perform as well as the self supervised gain for the task of representation quality which may really be the most in most important application of Ganz is whether you can use it too for unsupervised learning for classifiers object detectors and miscellaneous computer vision tests so this is the C for ten classification accuracy about 80% pretty similar to the others though on C part ten but on imagenet it's much different it's much higher than unconditional or rotation only and a little bit higher than conditional so this is the comparison of the self supervised game with other unsupervised learning techniques and this began model was recently improved with a big can model as well and does outperform this and it's just interesting to see all these different techniques that people are coming up with to make use of unlabeled data for representation learning so thanks for watching this video on self supervised gann begins please subscribe to Henry AI labs for more deep learning videos