Space-Time Video Completion | Two Minute Papers #137

Key Takeaways

dear fellow Scholars this is two minute papers with Caro today we are going to talk about an algorithm that is capable of filling holes in space and time classical image in painting or in other words filling holes in images is something that we've explored in earlier episodes there are several incredible techniques to take care of that but with this piece of work it is possible to generalize such a technique for video and F holes for not only one image but a series of images like removing an umbrella to create an unoccluded view to the beach or removing a waving person from a video of us jogging these results are truly incredible and even though this method was published long long ago it still enjoys a great deal of reverence among computer Graphics practitioners not only that but this algorithm also serves the basis of the awesome content to wear F feature in Adobe Photoshop CS5 in this problem formulation we have to make sure that our solution has spatio temporal consistency what does this mean exactly this means that hols can exist through space and time so multiple frames of a video may be missing or there may be regions that we wish to cut out not only for one image but for the entirety of the video the fieldin regions have to be consistent with their surroundings if they are looked at as an image but there also has to be a consistency across the time domain otherwise we would see a disturbing flickering effect in the results it is a really challenging problem indeed because there are more constraints that we have to adhere to however a key observation is that it is also easier because in return we have access to more information that comes from the previous and next frames in the video for instance here you can see an example of retouching old footage by removing a huge pesky artifact and clearly we know the fact that Charlie Chaplan is supposed to be in the middle of the image only because we have this information from the previous and next frames all this is achieved by an optimization algorithm that takes into consideration that consistency has to be enforced through the spatial and the time domain at the same time it can also be used to fill in completely missing frames of a video or it also helps where we have parts of an image missing after being removed by an image stabilizer algorithm video editors do this all the time so such your rest ation technique is super useful the source code of this technique is available I've put a link to it in the video description thanks for watching and for your generous support and I'll see you next time [Music]