Better Than DAIN? NEW BEST Tool for Boosting Video's FPS with AI [RIFE/Flowframes]

Key Takeaways

dane has been a really popular tool used for frame interpolation or commonly referred to as boosting a videos fps with ai to be more precise video interpolation is where an algorithm or an ai creates non-existing frames from pre-existing frames in order to get a smoother video experience but about a week ago i think something that is better than dain emerged and it's called real-time intermediate flow estimation short for rife which is also an ai research paper the most obvious difference can be spotted here in this video interpolation comparison and as you can see dean generates artifacts which shows it has trouble interpolating new frames well rife does not so while playing some montage of these videos being interpolated let me also blab about why this is particularly interesting the reason why dane is a bit special is because it considers the depth information within the video instead of like other simpler frame interpolation algorithms where it just takes the average between the previous frame and the next frame however the time required for dain to compute is too much of a drawback to just to get results that rife can get or even better in a much shorter time theoretically rife may be less advantageous when it comes to creating frames that has a strong sense of depth compared to dane but hey dane requires 10 hours to process a 2 minutes clip while rife only takes 15 minutes to process the same thing i see rife as an absolute win also practically i don't think the difference is noticeable and people probably can't tell the difference in between those split seconds an important note here is that rife is not just a simple frame interpolation method what rife does slightly differently is that it uses a neural network called if net that can directly estimate the intermediate flows from the images and is a lot more precise with the flow and the fusion process and consequently it works better faster and at a higher quality one of the problems that ai frame interpolation shares is that it cannot handle high resolution as an input so dain can only handle around 720p while rife can handle around 1080p so it is a fat w that rife can handle frame interpolation at a higher resolution and be at least 10 times faster than dana would say on the other hand there are also ai and non-ai methods which you can do frame interpolations with but the quality of the interpolation is not guaranteed to be as good also there's another really huge problem with frame interpolation fast and small objects tend to disappear or create flashing effect in the end result this is because the ai or the algorithm struggles at finding objects when it is traveling across the screen too fast you may say it needs to look at a wider range in order to find the coherent motion to generate the frames in between but if they cannot find the object because the difference between the frame before and frame after is too big then the generated frames would not have the object included which would cause the frames to miss the fast object which then creates this flashing effect this also goes for sketch animations as the lines are too thin the ai cannot coherently find where the line went so it just creates a blank screen in place of the line instead similarly artifacts between the videos are quite a common sight especially in anime anime usually has a really low frame rate due to the fact that each frame requires manual labor to draw so between a fight scene where there is a lot of motions going on it would be normal for a frame to be largely different from the next frame that is why ai generates artifacts when the motion cannot coherently transfer and unable to create a logical average frame in between to be fair this is still really far away from being incorporated into actual or even commercial use because it is still not the best at performing the task but this is definitely a good eye bleach from time to time especially for producing frames that are usually difficult to obtain like in stop motions or animes and this technology is only gonna be even better and better so just a few more papers down the line we will be stonks forever and thank you guys for watching this video is sponsored by infinite red infinite red consulting handles your mobile web and ai needs if you are looking for someone to build your app visit and reach out at infinite.red if you want to play with the ai yourself i made an installation tutorial linked below if you want to play it with it locally basically on your computer you can also run it online with collab too i'll also link that down in the description if you have any questions join my discord and big shout out to mark sven and many other patreons that support me on patreon follow me on twitter if you haven't and i'll see you in the next one