Eyes tell all: How to tell that an AI generated a face?

Key Takeaways

hi there we need to talk about gan generated faces after we spotted this paper beautifully entitled i still all ah when did paper titles become this sketchy i personally do not have any trouble with making research sexy as long as there is no all you need in titles anymore as you have already guessed this paper is highlighting that gan generated images at least with style gun too do not render the pupil accurately enough and therefore one can build a gan generated face detector upon this feature miss coffee bean saw this paper received an enormous amount of attention on twitter with reactions ranging from dismissing the importance of this pupil feature because the next gan generation will fix this problem anyway to cases of surprise that gans did not capture this human feature so what are the implications of this paper that highlights the pupil feature in particular and more generally how to tell gan generated images apart from real ones this is what we will discuss in this ai coffee break hey do you have your coffee ready because now we begin this paper here highlights a problem in gan generated phases gan stands for generative adversarial networks if you want a refresher of how gans work to produce images check out our previous high level explanation about this so the authors investigate how gans failed to generate circular or analytically shaped pupils for human eyes therefore the authors propose a method to automatically estimate the shape of pupils and determine if it is human-like enough since gans tend to generate weird pupil shapes this should automatically detect whether the depicted person is real or generated by again but miss coffeebean sees here two categories of problems let's go to problem number one the paper is framed like gans in general have this property of generating weirdly shaped pupils but if we look closely into the experimental section we see that the authors have quantitatively estimated this property only with style gun 2. so to generalize to gan generated in general is a bit of a sneaky exaggeration do not get us wrong here it is absolutely the case that many other gans have the same problems with pure pills too at least when we zoom in a lot from his coffee bean the pupil problem is not even that strong in this case which is strange because the authors reportedly used exactly the same cool website this person does not exist.com do check it out to retrieve some style gan two generations in any case gan generation quality is not there yet where details are rendered well just look at the eyebrows and eyelashes too these little details are easy to overlook if these gan generated images are used in settings where pictures are not looked at in full detail for example in bot accounts on social media where the profile picture is so tiny anyway so understandably there is need for an automatic bot checker as the authors propose but the argument of using this pupil detector widely grows weaker because style gun 2 won't be nvidia's last generation of gans ever and other parties are training powerful gans too especially since gan generated face detection is an arms race now that the defenders know of this pupil feature the attackers will do their best to make the right generations in the next version circumventing automatic gun generated face detectors so what we can learn from this paper so far is that when zooming in and looking at the tiny details we see problems in gang generations it's not really news that tiny coherent structure is hard to enforce look at the details of hair everything about the eyes teeth or tongue or lips for example miss coffee bean should we write the paper now lips tell it all i'm just kidding but coherent structure over larger distances is even harder especially for elements with highest diversity in the images in the training set we argue that pupils would be the first ones to be fixed in gang generation really because that feature occurs in most of the phases but structures spanning longer distances in the image that are only exhibited by a subset of the data is far more problematic just look at the background that is highly diverse in the training data and should be diverse in the generations too it seldomly makes sense unless it's uniform like where is this person even situated or look at the ears for example which are not often featured both in images because of the lateral angle or hair covering them up this picture was generated by style gun 2 2 and look how the ear lobes are at different heights but sure all these problems are not that easy for automatic detection with handcrafted features like the pupils are a funny side note the problem with the style gun generating bad pupils is weird perhaps unexpected but it's well known that neural networks tend to pay attention to very weird tiny features like tiny details between animal species when it comes to their fur structure while being sloppy on more obvious features for humans like learning that dogs usually have for legs they surprise us negatively as much as positively personally i was astonished when learning that neural nets even pay attention to visual chirality being able to tell whether an image has been mirrored or not by judging from things that are statistically more often encountered in one mirror ring than in the other like buttons on men's shirts for more check out our video now let's get to the problem number two of this method the authors acknowledge that detecting how elliptical a pupil is could discriminate against people who just happen to have not circular pupils the authors come with no real suggestion here to circumvent the problem and say that it is an infrequent phenomenon and abnormal pupils were not found in the real images of the datasets they used consisting of professional photographs that this dataset doesn't contain any abnormal pupils does not mean anything to miss coffee bean since this could be due to any kind of selection bias therefore she would like to stress the warning here not to use only an ipupil normality checker if at all otherwise imagine people being told sorry we don't allow you to create your tinder account because our anti-gann ai tells us that you have robot shaped pupils bye so what to conclude that humans given enough image resolution and knowing where to look for errors can still know if an image is gang generated or not but the gan that is making detection humanly impossible is not really that far away from now so we need automatic methods that detect those a method based on strangely shaped pupils is useful for now for style gun 2 and its likes but is short-lived like so many things in machine learning what miss coffee bean likes about this paper is that it puts things into perspective for non-machine learning experts let us know what you think about the paper in the comments and do not forget to like and subscribe [Music]