Creating Photographs Using Deep Learning | Two Minute Papers #13

Key Takeaways

[Music] dear fellow scholar this is 2minute papers with K here in this work we place a small light source to a chosen point in a scene and record a photograph of how things look like with the given placement then we place the light source to a new position and record an image again we repeat this process several times then after we have done that we have the question what would the photograph look like if I put the light source to places I haven't seen yet this process we call Image relighting this work uses new neural networks to do relighting by learning how different light source placements behave if you haven't heard about neural networks before make sure to check out our previous episodes on the topic I have put links for you in the description box after the training this technique guesses how completely unknown light source setups would look like in reality we give the algorithm a light source position we haven't seen yet and it will generate us a photograph of how it would look like in reality and the first question is okay but how well does it do the job up I am not sure if you're going to believe this one as you will be witnessing some magnificent results on the left you'll see real photographs and on the right reconstructions that are basically the guesses of the algorithm note that it doesn't know how the photograph would look like it has to generate new photographs based on the knowledge that it has from seeing other photos and it's completely indistinguishable from reality this is especially difficult in the presence of the so-called highfrequency lighting effects The High Frequency part means that if we change the light source just a bit there may be very large changes in the output image such a thing can happen when a light source is moved very slightly but is suddenly hidden behind an object therefore our photograph changes drastically the proposed technique uses ensembles it means that multiple neural networks are trained and their guesses are averag to get better results what do you do if you go to the doctor and he says you have a very severe and very unlikely condition well you go and ask multiple doctors and see if they say the same thing it is reasonable to expect that the more doctors you ask the clearer you will see and this is exactly what the algorithm does now look at this on the left side there's a real photo and on the right the guess of the algorithm after training can you believe it you can barely see the difference and this is a failure case the success story scenarios for many techniques are not as good as the failure cases here these results are absolutely stunning the algorithm can also deal with multiple light sources of different [Music] colors as you can see machine learning techniques such as deep neural networks have opened so many doors in research lately we are starting to solve problems that everyone agreed were absolutely impossible before we are currently over 2 ,000 subscribers our club of Scholars is growing at a really rapid Pace please share the series so we can reach people that don't know about us yet let's draw them in and show them how cool research really is thanks for watching and I'll see you next time