AI Converts Cartoon Characters To Real Life [Pixel2Style2Pixel]

Key Takeaways

for the past few weeks or maybe months you may have seen some pretty interesting images floating around the net that were labeled as ai generated you may have come across it gave a laugh and kept scrolling well the thing is have we ever seen something like this just maybe a few years ago yeah maybe you probably have seen some professionally rendered images or artistic works and a lot of unnatural images that seem rarely generated but it is getting harder and harder to tell if an artwork is done by a human or an ai and the fact that the ai research has progressed so much in such a short time is just crazy around this time a year ago video frame interpolations made its debut onto youtube not only that ai colorization joined the ai hype around nine months ago and video super resolution has always been at use for all these things and some of you that are new to my channel may be wondering how's this related to whatever clickbaity thumbnail i made for this video well in short these are all the work of ais including the one that i am talking about today around eight months ago a really famous ai research paper presented its sequel which is called stylegen2 it is basically an image generation model for realistic human faces producing state of dr results and it has been the basis for a variety of other image synthesis researches and i also covered quite a few of them so check it out if you haven't but by just producing state of the art results does not mean that it is fully utilizable and controllable to simplify one of the problems in style gen 2 it uses a few valves okay maybe not a few but generating realistic phases based on whether the valves are on or off the valves are kind of like the parameters that dictate how the faces will look at the end but the results for style gen 2 are really inconsistent so how much you turn each valve would effectively change the end results by a lot and even if you don't turn it the results each time will be subtly different so how can we accurately represent someone's face with just the valves this ai research paper called encoding and style a style gen encoder for image to image translation short for psp provides a solution where well of course it's not this easy but but it's like changing the basic valve head into something like an ultra high resolution electro pneumatic closed loop proportional pressure control valves where it can now be more consistent and precise at generating a specific face such as tom holland just a bit less attractive so instead of just playing with those basic parameters to try to generate someone that has brown hair and is a male this paper is able to use those parameters to represent how tom holland looks like so what does this give us remember the creative usage that i mentioned at the start of the video this ai research paper is able to improve that creative usage and create many more awesome image manipulations or image synthesis a lot better than many other older ai papers by having the valves accurately representing the faces so we can edit the facial features more precisely for example pulse has a technique where it uses downscale matching to find the super resolution of the input and the results vary a lot for this ai research paper it is a bit different from just downscaling and matching but we are able to encode crucial information about the face that was in the input image and produce a super resolution of it however the faces may still vary because it is impossible to accurately depict a super blurry face and what it'll look like when it's not blurry but the difference is really tiny and not only for face super resolution it also opens up possibilities for translating a sketch of a face to a real one which is really similar to a recent paper from sketch to face it can generate really realistic faces just by defining key facial features another really unique application is face frontalization which is similar to nvidia maxine it can generate the full face just by looking at the side profile but maybe this ai does not do as good of a job as compared to maxine so as long as the input phase has the crucial information right then that means it is time to bring drawn faces to real life again the results from nathan shipley are shockingly good the ai is able to pick up the facial features and pass these features into that super low name valve we used metaphorically and decoded the disney characters with a human look and not only it can depict the facial features of those cartoon characters it can also work on other realistic illustrated faces from league of legends the witcher final fantasy half-life gta 5 or even various famous paintings as you can see here it seems that in some cases the ai takes the wear and tear as a feature of the face so laura croft has some freckles now on her face it does not work on anime characters though because those faces are too exaggerated and simplified which is an amazing nightmare fuel i would say overall this is a really fun ai that has a really high potential of great creative usage not only artistically but also technically and lastly for the fellow ai nerds the major contribution of this paper in a less ambiguous term is being able to find the latent code of the real face inside the latent domain of a pre-trained stylegen2 model and i think you can also train it to encode and other pre-trained models too and this can definitely be a key to solving a wider range of image to image translation problems and it seems like we can expect many great things coming up in the near future so subscribe to stay tuned this video is sponsored by infinite red infinite consulting handles your mobile web and ai needs if you're looking for someone to build your app visit and reach out at infinite.red and hey you are at the end of the video thank you so much for watching till the end if you want to play around with this ai i'll link the collab down in the description if you are excited to talk more about this ai or share your funny results head over to my discord channel and as always i'll see you all in the next one