Image Restoration AI - Upscale and Restore Faces with DFDNet

Key Takeaways

photo colorization frame interpolation and super resolution have been a really hot topic recently they work really well as a media restoration tool for example videos such as a trip through new york city in 1911 demonstrates how well these technologies can perform in just under 8 minutes to expand on the field of media restorations today i am going to introduce you blind face restoration via multiscale component dictionaries short for dft net this ai provides solutions that most super resolution would struggle with for instance image super resolution ai like waifu 2x normally just upscale the entire image which in hindsight may serve its purpose but actually still leave out some problems like creating the details that were never there before which is impossible and remember that polls video where it can in a way upscale images from pixelated faces or super low resolution well it basically used a matching technique to guess the upscale resolution with style again which is a really fascinating way of creating new details so similarly what dfd knit does is that it segments more visually detailed parts of the face like the eyes mouth and teeth and guide the upscaling ai by matching these similar features in low resolution and use the train model to provide a matching guess of how it would look like in higher resolution and clearer details and this is called reference based phase restoration so to clarify the generated details may not necessarily mean someone has a white mole on top of their head it just took the whole image and without the context that it might be a damage printed photo and the same goes for the pupil color too maybe because of the reflection the eyes would turn out brown instead of blue here while normally pewdiepie's eyes are both blue so not only the less significant visual details are upscale but also the highly complex visual details are created at the same time which surpasses what normal image upscaling can do and it produces more convincing facial upscale results in my tests i stole some images from our slash old school cool and attempted to restore faces on different kinds of old photos from color to black and white a really obvious takeaway here is that ai cannot pick up really distorted faces faces with too much exposure profile views and blurry basic facial features and structures probably will work well on poles though probably will work well on poles though okay while following up the colored images most of them are really successful at upscaling the faces and it's generally really satisfying looking at all of them however the physical damage of the photos may be considered as facial features so terry crews in his teens may look like terry coos in 2019 and look at this even this can improve pewdiepie's camera quality and look at his beautiful blue eyes just like every other test results it just shows how useful are these segmentation and estimation when used in photo restoration for black and white photos this ain't colorization ai so it makes sense that it just outputs the original color it does what it's supposed to do and everybody likes that and here's an interesting test which one would be better upscaling before colorizing or upscaling after colorizing and as i speculated colorizing first then upscaling will work better and become more natural as sophisticated details are able to be regenerated accurately and upscaled and if you clearize it after it upscaled the colorization might not look as precise as upscaling first that's it for today's video if you want to know how to install this ai i've made a video on my second channel and join my discord if you encounter any problems or you can just join without any problems i also recently started a patreon check it out along with my twitter if you are interested thank you guys for watching and i'll see you all next time