GANs for Good- A Virtual Expert Panel by DeepLearning.AI

Key Takeaways

hi everyone and welcome my name is ryan keenan and i'm the director of product here at deeplearning.ai we really appreciate you taking some time to join us for this event today and we sincerely hope that you and your families are doing well this has been one crazy year we have people joining us today from all over the world in fact i believe it's more than 140 countries joining today which means that you're in pretty much every time zone so good morning if you're in california like me or good evening good day good middle of the night to you whatever it is where you are whatever time it is we really appreciate you joining us taking some time uh to join us for this session so at deep learning.ai our mission is to make world-class ai education accessible to everyone and so with that we aim to empower you our community of aspiring and current machine learning practitioners to be part of the ai transformation that's happening across so many industries part of that work for us is building new courses and today we're excited to be launching a new three-course specialization focused on generative adversarial networks or gans for short gans are an emergent technology within the space of deep learning and have been used for a wide variety of applications in recent years most famously you might be familiar with how gans are being used to generate really realistic looking images whether that's pictures of people who don't exist or other things but gans have also been used in applications that you might not think of like improving cyber security or even generating music and so this uh success of gans in so many different applications has also come with some controversy because with this powerful technology it becomes possible to do things like generate deep fake videos of people saying things they never actually said uh and other things that make people wonder whether this is a technology uh that's a good thing or a bad thing so with these courses that we're putting out today and with this session we're focusing on how you can use this powerful technology in exciting applications that'll have a positive impact on the world so for today's event we have a live panel discussion and we've assembled an amazing group of gans experts they'll be discussing some of their current projects uh the importance and the future of gans and then also providing some practical career advice for those of you who might want to break into the field so our agenda today will be first off keynote speeches by ian goodfellow from apple and anima anand kumar from nvidia following the panel discussion uh following rather will be a panel discussion where ian and anima will be joined by alexey efros from uc berkeley and sharon joe from stanford and of course our very own andrew eng from deeplearning.ai after the panel discussion we'll do a q a session and we've also lined up a course demo with uh sharon joe the instructor of the courses at the end of this event so stay tuned for that for the q a session we'll be selecting questions from the youtube live chat so we recommend you post your thoughts there and your questions and that way if we see a lot of people asking about the same thing or if we see something that's particularly relevant to the conversation then we'll go ahead and surface that to the panel in the q and a session and now without further ado i'd like to welcome our first speaker ian goodfellow ian is the director of machine learning uh at these in the special projects group at apple and previously worked as a research scientist at google bring basically all of us are here today in this event because back in 2014 ian published a paper and brought gans into the world so it's a great honor and a pleasure to welcome you to this forum today ian and i'll turn it over to you now thanks ryan and welcome everyone thank you for joining i'm going to kick off the event with a quick summary of how gans work and i'll also show you how some of my colleagues at apple have been able to apply them to some exciting applications um gans are designed to solve the generative modeling problem generative models take a collection of training data like these images from the cell of a data set shown on the left here and infer which probability distribution generated that data then the model can learn to do one of two things either it can learn an explicit density function describing that probability distribution or it can learn to create new samples from that distribution as shown on the right here where again has created two faces of people who look like they could be celebrities in an imaginary world they come from the same probability distribution as the other celebrities but they never existed before they were created from scratch by the model the way that gans learned to do this is using a game theoretic setup where two different players seek out a nash equilibrium of a game one of these players is the discriminator it looks at real images like the moth on the left and is trained to assign high probability of the moth being real rather than fake the other player is the generator network it produces data such as the photo of this cow-like monstrosity on the right side of the slide the discriminator looks at these fake images and attempts to learn to assign very low probability to these images being real the reason that this is a game rather than an optimization problem is that the generator simultaneously tries to output images that the discriminator will assign high probability of being real the nash equilibrium of this game consists of the generator producing perfectly realistic models images that are indistinguishable from real so the discriminator will output a probability of one half of everything being real over the first four and a half years after my colleagues and i introduced the first gen paper institutions and people around the world came up with new ideas to improve the performance of gans and in about four and a half years they went from able to produce only small low resolution grayscale images of faces to being able to produce very high resolution photorealistic faces likewise when i first had the idea for gans they didn't work very well for producing a wide variety of different objects but over about two years starting in 2016 a variety of people came up with ideas that helped guns go from struggling to produce low resolution images from the imagenet dataset to being able to produce high resolution images of all 1000 imagenet categories now that gans are working really well it's interesting to study them and learn how to apply them to different tasks um one task that i've been especially excited about is using gans in the physical world for medicinal applications for example glidewell dentistry has been able to use them to make replacement dental crowns the gan needs to actually make a crown that fits the shape of the other teeth in the patient's mouth and is able to bite and chew so it has to be both personalized and functional traditionally it took a human technician about two weeks to do this so the patient would need to have their tooth prepared for the crown and then go home with a temporary crown for about two weeks and then come back with the permanent crown to be installed with again it's possible to 3d print the new crown on site and get everything done in one day at apple some of my colleagues have found other applications afghans one of these is scans for augmented reality in augmented reality we often want to insert objects into a real scene such as this teapot inserted into a scene with a table in a notebook in the upper left we see what it would look like if we rendered the teapot with reflections showing only things that are available in the incomplete environment map shown on the lower left this is the part of the environment that the camera can actually see when the camera is facing into the scene as you can see in the top of the incomplete environment map the camera has no way of observing anything like the ceiling the walls the lighting of the room this is where gans come in we can fill in the missing part of the environment map as shown on the right this isn't just extrapolating pixels from what we can see it's actually using our knowledge of the statistical structure of the world to understand things like that the ceiling tends to be relatively plain and that lights tend to be near the ceiling given this completed environment map we're now able to make much better rendered reflections as shown in the upper right here's a video of what it looks like when we insert this instant pressure cooker into a scene you can see from this angle that the reflection is capturing objects that are actually visible to the camera but as the camera moves around you can see that the reflection is pleasant and has an environment that spans the whole room not just what's visible to the camera several other applications of gans at apple rely around generating training data for other models the first published work in this direction won a best paper award at cvpr2017 a lot of people have asked me if you can use gans to make training data for classifiers by training the gan on a data set and then just producing more examples from that same data set as far as i know that doesn't work you would need the gan to generalize well and produce new examples that show new patterns in just the same way that the classifier would need to understand those patterns in new input examples what you can do with gans instead of just making new data from one source is to meld two sources of data in this example we want to train a model that can tell the direction that an eye is gazing so that for example you can highlight an icon on a phone screen just by looking at it using unlabeled real images is difficult because it's hard to add a label saying where exactly the eye is looking instead you can use synthetic images where you 3d render eyes such as the ones shown on the lower left the challenge with this is that it's not a particularly realistic image after you've 3d rendered it you know where it's looking because you positioned the model yourself but the appearance is not quite good enough to actually train an eye guys direction classifier gowns can be used to learn from the unlabeled real images and make a refiner that upgrades the synthetic image to look more realistic like the refined image shown on the lower right so the gown has melded two sources of data real photos and their realism and synthetic photos and their labels so we can get realistic label data at the end we also use this same idea in a recent application for quick path quick path is a text entry feature for ios and ipad os where the user swipes out a series of letters in a single finger strip without lifting their finger here you can see three different ways that people swipe out the word anybody with a trajectory starting on the letter a and then turning from green to yellow over time as the trajectory is completed these trajectories need to be recognized with a neural network because they're hard to recognize with handwritten code over the course of the trajectory the finger visits many keys but only a few of those keys are actually meant to be included most are just being passed over in order to train this classifier we can just gather data of real trajectories made by real users but we can also use again to combine sources of text with that a smaller trajectory data set and make new trajectories for new pieces of text melding those two together allows us to train the model on a lot more data one common way of looking at how machine learning models scale with more data is to make a log log plot work it by do show that these loglog plots often look like a straight line where as you move to the right across the the data set size axis describing the number of examples in log scale we see that the error decreases linearly if we also plot it on a log scale the blue curve here shows that if we train our quick path trajectory recognition model on real data we do get one of those linear curves if we then switch over to training using new data coming from again the error continues to decrease but with a different slope in the log-log plot from this we see that again example is worth roughly one-fifth of a real example fortunately it costs less than one-fifth as much to collect because it's easy to continue sampling from the can so this becomes a very efficient way to train our trajectory recognition model overall i think these strategies for generating gan data can be useful for a lot of socially good applications you can generate a gan that is differentially private and then it can make fake medical data for users researchers to use without compromising any patient's privacy gans can also be used for fairness accessibility and inclusive personalization by over sampling portions of the data set that correspond to underrepresented groups or groups where it's hard to capture data for that population i'm looking forward to seeing what the other members of the panel have in mind for how we can use cans for good thank you thanks ian that is some really interesting stuff i i i find it uh cool that you were able to quantify exactly how much gan generated uh training data ism is improving the model compared to more real examples i think that that's something that i've kind of wondered about like i'd heard that you can generate more training examples but like how good is it or how good are they really compared to being able to to get new information but but then being able to say okay well given the cost this is a much more efficient way to to improving the model very very cool stuff yeah i was really excited to see this this paper from my colleagues yeah awesome uh well now i'd like to welcome our next speaker anima anan kumar anima is a director of machine learning research at nvidia she's also the brand professor of computing at caltech today she'll be sharing with us the role of interaction and disentanglement in the training of gans welcome to this session anima it's really a pleasure to have you with us i'll hand it over to you now hello hi everyone thanks for having me here and it's really a pleasure uh uh to be part of this event and thanks ian for giving such a great overview of gans so today i'll give you some of our recent work both at celtic and nvidia and how to understand gans better you know there is complex interactions happening between the generator and discriminator how can we make more sense of it and can we design better optimization methods for training gans that'll be the first one i'll introduce and the second part is how to understand disentanglement in gans better you know the example that ian showed about the dental application there may be so many different properties you may want for the teeth figure so how do you ensure that you can separately control all those different style factors and that's something i'd like to go a bit deeper into so that'll be the two agenda for this talk today so the first one appeared at icml this year and the question was can we think about the role of implicit regularization in training gans you know for standard supervised learning uh it's now for fairly common knowledge that implicit regularization of stochastic gradient descent is responsible for getting good generalization and for being able to train models such as the imagenet uh data set so okay is what is the counterpart for gans when it comes to the role of optimization method and this is actually very important because ultimately what we would like the gan to do is have a discriminator that can really separate the real images from the fake images right i mean those images that don't look realistic for instance this data sample here is not within the image distribution so you want the discriminators to say that this is not a real image but then the discriminator could potentially cheat because of the uh minimax uh game that the generator and the discriminator is playing the discriminator could get uh really good around the training data points right so it can just get very peaky here meaning just around the neighborhood of the training point it can say this is real or fake and everywhere else you know it can just say that this is not a real image but this is not happening because um you know if we are not constraining the discriminator it can get perfect but we are going to regularize the discriminator right so if you've trained any of the gan models you know to regularize the discriminator and uh the best way to do that is to uh do some penalty on the gradients so given that we are adding this regularization we are no longer talking about the original min max optimization between the generator and discriminator and uh so for this we did a very simple exercise you know we trained gan to a good checkpoint and then we wanted to continue training so we could do that either by simultaneously updating both the generator and the discriminator so doing gradient descent on both discriminator and generator or only update the discriminator right and as expected if you only update the discriminator its loss goes down so the discriminator can get really perfect on the other hand by training simultaneously with the generator it's not able to get really good and so that's what we call this as the implicit competitive regularization because this competition between the generator and discriminators simultaneously updating their weights is responsible for it not to lead to pathological solutions and so the question is can we further improve this interaction between the generator and discriminator and lead to good stable training of gans and so that's where we introduce a method known as competitive gradient descent and the main intuition is can we go beyond just simultaneously each agent updating gradients on their own and introduce a term that models the interaction between the agents and just to introduce some minimal math here if you are just doing gradient descent individually on both the discriminator and the generator that only looks at the gradients right so there is no interaction that is being modeled directly between the players on the other hand if we now want to ask how to incorporate interaction we can now look at this taylor series expansion of the objective function and introduce a term that captures the next order interaction and that would be the mixed hessian between the two players so we now have the taylor series expansion be a little more accurate but at the same time this is still a good structure in the equation because it's only a bilinear interaction in the two players so it ends up now viewing the game between the generator and discriminator as a local bilinear game and so once we can make this taylor's approximation we can in fact solve this in closed form i wouldn't worry about the exact form just to showcase that what we are now doing is a modification of gradient descent right so in gradient descent each player would only update their own gradients but instead we are introducing certain correction terms based on how they're interacting with the other player and intuitively what this gives us is more stabilization because you're also looking at if you're the generator what the discriminator is going to do you know in which direction is the discriminator moving and you'll account for that while making the moves and what we see is if we uh now tried this on gans we get uh state-of-the-art results in terms of the inception or fid score but with no penalty right so the intuition is you don't need explicit regularization if the implicit regularization is strong enough to accommodate for stabilization between the two agents and that's where the uh introducing the new optimization method can potentially now help us to train more in a more stable way and uh tuned better for training guns and this is not just for training gans we also see that in reinforcement learning environments for instance this is a racing game where both the agents have to learn from scratch on how to win the race right so the one that is faster wins the race and if both were just doing gradient uh updates on their own you can see that it leads to poor strategies right so first of all there is a big gap between the two competing players and even the best one isn't that fast as the one where you incorporate uh the interaction term in terms of the mixed hessian and design policy gradients based on this interaction and so what we see in these examples is that rethinking optimization in terms of the dynamic interactions between the agents either in training gans or in this racing game that i showed you can help us lead to better outcomes we can train now gans that are more stable during training and lead to better uh image quality uh without the need for a lot of other external uh regularization that would entail additional tuning and so this is one aspect of understanding the dynamics of gans and not just thinking about the equilibrium point of what the gan represents as a mini max game so the other aspect i wanted to cover was understanding disentanglement in gans and in particular stylegan which is a popular architecture that's developed by nvidia researchers and so when i talk about disentanglement what we really want is different style factors you should be able to individually control right and humans are great at this you know we have different concepts we've learnt as infants and we've done that in an unsupervised way and this way we can now compose and make entirely new images or new concepts and so disentanglement is the key for compositionality so if you're looking at the style gan architecture the idea is you also now have style factors or these factor codes that give instruction on what property do you want in this image right so like the example that ian showed with the dental case you know what is it that you want functionality in that image to be represented and we have both the versions where you may generate an image from scratch or the editing version where you can input an image such as you know the one of professor fei-fei lee and then you want to control or change the characteristics of that image and it turns out that if you were doing this in a completely unsupervised way that's challenging right it's very hard to challen have the different factors say of the mouth eyes and so on be completely disentangled and so what we came up with was a semi-supervised approach where we could have a very small number of examples that have the labels you know that say whether the mouth is open or not and if you can then combine together and design good losses to encourage disentanglement then we can make use of this supervised data and that's the results we see on the celeb data set you know the same data set that ian also earlier showed we can now introduce all kinds of attributes such as banks or glasses and do that in an effective way with as little as point five percent of the label data so of the total training data if as little as point five percent is labeled that is sufficient and more labels actually didn't improve uh disentanglement so it seems like you know as long as you have a little amount of labeled information that's enough to encourage good disentanglement and good image quality and we also see that uh with also you know any uh external images that were not in the training data be introduced and uh you know we see that it's able to also do this in the editing mode but again with training being having very little amount of label data just to quickly kind of understand the losses that go into encouraging this disentanglement the idea is now to combine unsupervised and supervised learning together right so what we mean by unsupervised learning is here the real image doesn't have the label so the only way to encourage disentanglement is on the fake images so if you now generate a fake image say with the mouth open you want to make sure that the encoder in the discriminator can tell that the mouth is open right so you're now checking between the discriminator and the generator that they're consistent with each other and for those that have the labels you can just directly check if the reconstruction of the style code is being done at the encoder in the discriminator so combining these two now leads to effective disentanglement and also good image quality and that's the idea that you know sometimes you may want to have a little bit of label data to encourage this disentanglement and you can get uh significantly better results so i want to conclude by saying that there's been a lot of exciting work in gans recently and it's important to think about the underlying dynamic interaction between the agents and we designed a new method known as competitive gradient descent that can better stabilize this interaction by incorporating the mixed hessian term that looks at what the other agent is going to do and this shows that you can even sometimes do away with explicit regularization and still get good results in training gans and also in other applications such as competitive reinforcement learning problems with multiple agents and the other aspect i showed was on looking at disentanglement so not only tr generating images but a controllable generation with various different style factors and here if all the images are unlabeled uh disentanglement is ill-posed and really hard to get at the right answer but as very small amount of supervision can greatly enhance this disentanglement thank you thank you anima wow there's a lot there i i uh in thinking about competitive graded descent it sounds like just an amazing step in the direction of the learning from each other thing it's not just that the prediction that's it's actually through the through the process of gradient descent as well that's that's incredible um next up we'd like to continue the conversation by welcoming our other panelists today these are alex a efros professor of electrical engineering and computer science from uc berkeley andrew eng founder of deeplearning.ai and our moderator for the panel and the instructor of the courses the of the gan specialization that we're releasing today sharon joe sharon will be facilitating the panel discussion today so uh just a reminder to everyone watching if you have questions for the speakers or just thoughts about all this stuff you can post them in the youtube live chat and we'll be selecting questions from there for the q a session after the panel discussion so sharon i'll turn it over to you now thanks so much ryan and if what you saw in the keynotes caught your fancy or went over your head that's what the specialization is here for and you'll get to learn from the very basics of what is again to the state of the art and style gand that anima had shown so with those fantastic keynotes from ian and anima let's start off by letting the new faces in the room alexi and andrew say hi to our audience alexi hi [Music] well done alexis [Laughter] hi i'm i'm i'm elixir first i'm i'm a professor in um at uc berkeley doing computer vision and um computational photography and um i've been a huge fan of gans i i don't fall in love with papers often but this was definitely love at first sight and so i'm yeah i'm really excited to be you know part of this of this canned story here and thanks to all of you from all 140 countries and all the time zones so joining us middle night morning evening um it's really exciting to be here yeah i had the pleasure of knowing knowing ian for a long time it's great to reconnect with him and of alexi whom i think i first met when we were both um uh phd about the entry phd programs and we ran into each other touring different university campuses uh and and i'm really grateful as well that anima and sharon are here um gang's a big movement you know it's one of those amazing technologies that frankly wasn't working so well six years ago when he didn't publish his first paper but now has really taken off thanks uh thank thanks uh the work of really nvidia and many other groups around the world and if music is poised to find a lot of exciting applications and i was really struck by ian and the numerous comments on the concrete use cases where gans are no longer you know something that generates cool pictures you look on the internet to things that are really useful and not going to people's mouth in the dentist office so i hope that some of you are watching online today will take the gan specialization and learn about these tools and go help build amazing things they'll go make life better for a lot of people thank you alexia and andrew uh for the next 30 minutes or so i'll be asking some questions to our panelists and while i'll be directing questions to a specific person i'd also like to encourage any of you panelists to jump in and offer your input in the conversation so starting with alexi uh can you tell us a bit about what your lab has been working on recently students in the specialization will be spending a large chunk of time learning about work from you and your students um sure i i as i said i'm a big fan of gans so that um you know there is a lot of things that that happen in in gans thank you ian for uh giving a shout out to our uh work with gladwell on the on the dental reconstruction this was done with my wonderful colleague stella yu um uh we uh lately we have been really thinking hard about disentanglement just like anima and because i'm i'm really focused on unsupervised learning we have been looking at at this endowment in an unsupervised setting and we have a several papers on that trying to use a hessian penalty for this entanglement and also uh thinking about using contrastive learning uh for kind of disentangling texture from structure we have a couple of very recent papers one is uh is on kind of replacing the cycle gan cycles in cycle will construct contrastive learning um that was in eccv uh and we have a just brand new paper at europe's called swapping out encoders where we are kind of separating the style and the content yeah uh again in an unsupervised way that we're pretty excited about apart from that i i've been working for a long time and self-supervised learning again you know going away from i'm i don't like labels so i'm trying to stay away from labels and i've been also pushing against uh i also don't like data sets even though i i love data but i don't like the fact that they're they're stationary and and and so we have been really focusing on uh online kind of streaming data learning and so we have a paper on um on on test time training where we're basically like uh adjust adapt to the streaming data in an online way and so i think but this is just starting and i think there's a lot of cool directions there yeah fantastic i love your contrast of learning paper and uh uh we are also in our lab with andrew uh looking at disentanglement so that seems to be a trend here in a sense i think gans are doing a lot of it just in segment even if you just just take the you know the bread just off the shelf style gun or big gun there is a lot of cool disentanglement there already and we just don't quite understand you know how much is in there so i think this is this is one of the exciting directions now yeah yes very much so and a huge trend in research right now uh speaking of trends uh andrew what trends around gans are you most excited about by the way not sure if i should say that sadly uh our submission to neuros on disentanglement did not get accepted but that's okay you know you you happens to happen to everyone you live and you move on and then you try again um excited about trends i'm excited about you know i think that um there's so much basic research on gans that's still going on and that's great right clearly the technology innovations new new modes still going on is great what i see is um in the in tech deep learning world over and over a patent you see is this new technology it works amazingly well in the lab um and this opens up the door for a lot of exciting creative applications i think you know maybe 10 12 years ago we started to see this for supervised deep learning and then we saw a lot of dominoes topple deep learning has its first major impact in speech recognition followed by computer vision image net and then i saw same transformation in nlp and and so on and so on and again patent matches to me to one of those amazing those technologies and that works so well yoga pictures you go wow this is so cool um but i'm really excited about exploring all the ways to take these and put them into into useful uh uh applications which is actually really difficult the whole ai world is not great uh frankly at bridging the research the proof of concept to the production gap but when ian talked about um the gaze tracking you're mapping one eye the computer graphics i to another one it turns out that in order to apply that broadly across many industries and many problem problems i think there'll be a lot of important stuff to solve right what happens if the mapping doesn't work how do you make sure you don't have mode collapse or training problems or all of those are issues not just when uh ian is at the one at the keyboard working at making it work well but how to make it systematic so that many teams hundreds of thousands of teams across the industry um can make it work and i i'm also excited about the creativity that gans could unleash everything from you know photoshop 2.0 uh uh to all the ways of all the ways we have to manipulate images i hadn't realized um apple had such cool work on ar using gans but i think all the creativity we need more people to understand these algorithms so that they can be the creative ones come with the cooling applications [Music] thanks andrew and i think anima also has maybe shares that uh perspective since she does spare uh spend her time between nvidia and caltech uh animo what are some of the unique challenges of applying gans to business that you see even if you're at nvidia and have a seemingly infinite supply of gpus well you know it's it's really exciting to be working at nvidia but like everyone else of course you know we want to make a good case for why we use gpus right and use it for good benefits to humanity and and that's where i'm very excited by you know by my colleagues at nvidia who've been pushing so hard and making uh gants photo realistic you know get to these really high quality images that have now passed the touring test and whether it's tilegame gogan has i think now maybe even up to a million downloads where you can turn into an artist and you can you know give like rough broad brushes of the landscape you want to draw and then it turns it into a really impressive image and then also moving more into the 3d world you know how to do generation there what are challenges and how to do it at scale that's where there have been a lot of great researchers at nvidia look looking into these cutting edge topics in generative models so more on the caltech side what i'm excited is looking at interdisciplinary research right and especially talking to neuroscientists like such as doriso and looking at how we can get inspiration from how our brain you know does this you know does the brain have a generative model apparently that question is still open and unanswered but we do have some form of feedback we do have some form of representations of the world around us and we when we are seeing and perceiving we are hallucinating uh to get to the actual image we are not just taking an external input from the world we are also building based on our internal representations and uh doing it as a feedback and so some of the recent work we've been looking at is how to add this kind of feedback to any standard feed forward neural network architecture and that can be a lot more robust to all kinds of corruptions that it hasn't seen during training because that's the other important aspect to bring it to the real world that these models have to be robust the current brittleness that even if you have a small amount of imperceptible noise it's completely going to throw off your predictions is not uh one that can be brought into the real world and so looking at how we can get inspired from the mechanisms in our brain and trying to bring some of that into uh neural network architectures like feedback is something that's been very exciting very cool and very salient uh going off of kind of the utility of gans uh ian can you expand a bit more on some of the topics you mentioned on using gans for good from your keynote particularly pertaining to perhaps differential privacy or competing bias yeah and a lot of these are areas that have only had little nibbles on them in the research literature that i'm hoping people from this event go on to explore in more depth one of them is differential privacy the idea behind differential privacy is you can train a machine learning model in a way where it doesn't memorize the individual characteristics of individual examples in the training set so if that model your training is again it can then make you new data without revealing anything about the real data that went into it at the start that's super powerful for medicine we've seen a proof of concept from casey green's lab where they actually trained gans on medical data and then they're able to make new fake medical data that can be released publicly and effectively an infinite amount of it especially because it's really hard to pool data from different clinics because of things like hipaa considerations things like differentially private generative models seems like a really good way to get over the data scarcity problem in medicine for a lot of other topics there's reasons why you might want to generate more data for a given area with again if you want to for example support a language that isn't spoken very commonly it might help to generate more data for that language or there's a startup called view.ai that allows people to visualize themselves in clothes available from a retailer traditionally you'd have to rely on the retailer having hired a model who looks kind of like you and now you can use again from view.ai to generate somebody you know who's your your race your skin color your hair color your body shape um it makes the whole model photography process a lot more inclusive in that sense so i think there's a lot of different things that you can do those are just a few things starting to scratch at the surface thanks uh i think anima had touched on this a little bit uh before in terms of how gans there's still a gap between gans in the real world and decision making environments uh versus and understanding where they do learn and are effective and where they might fail uh ian did you want to touch a bit about this yeah it's it's also pretty similar to what andrew said about how supervised deep learning went from the lab to the real world the state of gans today kind of reminds me of the state of supervised deep learning maybe like circuit 2012 that it used to really take a wizard to train a deep learning system and to some extent today gans are still like that now deep learning is considered relatively reliable and it's because we found all these nice recipes like always using release always using momentum maybe having a few technologies that didn't radically change the paradigm but made it so much more reliable like atom and resnets i'm hoping that we get those kinds of reliability technologies that help us to apply against in lots of applications without needing a again wizard yeah one of the interesting things to me it's just really going off ian's comment what do you think seems to me is that um a a problem if i may about about the whole gan world is we don't have very good evaluation metrics and so we can generate it either ooh this looks great um in fact one of the pieces of work that i uh i found really interesting was was some of sharon's work on hype uh showing you know how challenging it is and how problematic it is to use some of the automated evaluation metrics and i think i think this contributes to the need for gang wizards because the right wizard looks at it and their eyeball says oh i gotta you know it's clearly more collapsed i'm gonna do this or something actually sharon don't mean to preach on the spot i know you're the moderator but since you're a world expert on this do you wanna do you wanna say anything about about this problem i think evaluation is a big problem in gans and you'll get to learn about it in the specialization but because it is a problem i think it very much depends on your downstream task and what you want to use your gan for um so your gan can help your other ai models which is cool but then you can evaluate your game based on how much it does help your a uh downstream ai model like classification segmentation um and the like uh or it could be around realism and then i think it's really important that we have humans in the loop um who who we humans uh are the gold standard perhaps of uh evaluating realism and in terms of democratizing uh gans which ian was talking about i think uh some of the really interesting work was the work by the nvidia group uh open sourcing stuff so the rest of us could use it so i i've gone to the nvidia website and then looked at anima yeah and i'm happy to announce just a few days ago we announced the release of imagineer so that's all the you know very sorry sophisticated and cutting edge can models we've now put them all in one place you know mingo's group has worked hard to have a hackathon for the past six months and get all that in great shape so that everyone can use it so i would love for more people to check this out through this you know throughout this course and maybe adopt some of those models very cool thank you i'm really excited about that and was following the the tweet on that pretty closely um maybe switching gears and thinking about our audience members a bit um and thinking about what they can do to prepare um to be a good gan researcher or practitioner um or student in general uh alexi maybe to start what makes a good ai researcher in your lab and what characteristics what characteristics do you see in your students that that you like well i mean my lab is is uh i'm pretty happy to say that all the main characteristic of my students and and me also is we're all a little bit crazy i think it it's kind of uh important to be a little bit crazy if you want to do research and i think really it's it's the same things that you know with any discipline you want to you know you know you want to be imaginative you don't want to focus on short term you don't want to have you know focus on getting papers out every you know three weeks um i think it's all about having some kind of thing that you really really want to do and and just just trying to go there i what i told my students is don't stress out about all these papers this this you know the the the faucet is open on archive and like there's so many papers coming out every day there's so many papers being published you know there is there is this idea in in in medicine you know when they graduate from los from medical school they tell you uh there is a concept of half-life of knowledge so they say okay you know remember five years from now half of what we have taught you in medical school will turn out to be false right just because you know science moves forward and i think in ml and in gans that that number is maybe three months half life of knowledge in gans is about three months so i would not worry about every single paper every single thing every single trick you know it's fine you can wait a little bit and see if it actually works if it sees if it takes off i don't read every paper if like for example you know if the paper only has faces in it i don't treat it because we know that faces are easy faces work 20 years ago using active appearance models so if they need to try something harder or if they just have you know mnist or something see if i you know i wait until they try it on harder data sets so that kind of cuts out a lot of the chatter so i would just not worry and not be stressed out because if you have a good idea it you know just go for it it will come out definitely and uh i i will say i have been scooped before i think i was thinking about doing a semi-supervised gan at some point looking to nvidia but um uh anima as a fellow woman in ai i and others definitely look up to you for advice and inspiration and it's without a doubt that you've lended incredible support to your students and your role model to a lot of women looking to go into ai what advice would you give to the women and girls tuning in tuning in and who are learning ai thank you sharon and yeah i'd love to see more diversity and inclusion in our field i think there's been incredible awareness in the last few years and uh you know the majority has been supportive and positive right so despite all the trolling and the some of the negativity i think the positivity overwhelms at the end of the day and that's why i continue to speak out and i continue to make sure that we create a healthy environment for everyone in the community and so what i would say is to keep fighting you know if you are seeing a problem uh you know whether it's a technical problem or it's a societal problem right it's something you care about you may not see the immediate returns you have to you know not only fight for it also find the allies find your support network and also learn how to communicate well because whether it's a research aspect or right everything else we want to see change in the community i think at the end of the day we need that support network and we need to make sure that we create the awareness definitely and on the business side for those tuning in on the business side as a successful ceo founder and professor andrew what's your secret in managing your time and keeping yourself updated on ai and what kind of advice would you give to folks tuning in from the business side should they take this specialization yeah actually but for fun i'll answer before just just a shout out to a nemo for really consistently speaking up on social media's issue that that matter i feel like we live in a world where there are you know lots of ups and downs and and i i guess uh watching the u.s political u.s presidential debate just last night i think it's more important than ever uh that when all of us have a strong opinion on something right or wrong that we speak up because i think every voice matters i think anima has been a consistent voice in that uh and then i think that's great and i think sharon you know uh release a app on the internet uh to help anonymize protesters you know to protect them i think it's wonderful that sharon just woke up and said this is social cause i'm a deep learning researcher i want to build this because i can make the world a better place i think every one of you watching this online your voice matters don't ever think that you know individually our voices are limited but collectively we the ai community are incredibly powerful but only if all of us speak up um about the things that matter so they can shift the whole world toward doing more the things we want to make people better off um and and and so and and i think um and so to answer shannon's question so much is happening um uh it really honestly one one of the one of my honest sources of knowledge is the badge uh which deeply dia publishes um there's a large team of writers uh that that are editing chief ted greenworld organizers to try to synthesize the most important ai news uh to share in a very succinct form with everyone um the huge the issue that's going on later today of the badge is against special issues so covers a bunch of cool research on ghan some of which i actually honestly did not know about myself until you know ted and his team found those stories and synthesized them so i should use that and i think unfortunately also to friends um my my my just to complete honest my personal number one source of knowledge on gans is actually sharon uh she sharon tells me when she sees something cool so so that's that's that's fun but i find that um having friends you can talk to the read papers together to brainstorm together uh it has been a very important part of um how i keep up to date and so i encourage others you know it's a formal community none of us none of us should have to do this alone so make friends form a community and work with them because we're all really much better and much stronger together thank you that's powerful uh the newsletter the batch actually has an interview with ian as well in it which i hope you all check out uh and i think this would be a question for all of you on how do you stay up to date on the latest machine learning research uh maybe starting with ian yeah at apple we actually have an event called paper party where once a week we have people get together and share a paper and i think that's really powerful because only the person presenting it needs to have read it it's hard to get everyone in a discussion group to read a paper but one person can explain it succinctly to other people and then beyond that just like how andrew relies on you for updates about gans i have friends in different subject matter areas that keep me up to date i actually read very few papers i i mostly discuss them with other people and that's and through events like paper party that's how i stay up to date great i know my my favorite kind of paper group was definitely one where there were no expectations going in to have read a paper uh we just read while you're the