Seeing Subtle Motion in 3D
About this lesson
3D Motion Magnification: Visualizing Subtle Motions with Time-Varying Radiance Fields Brandon Y. Feng* (University of Maryland College Park), Hadi Alzayer* (University of Maryland College Park), Michael Rubinstein (Google Research), William T. Freeman (Massachusetts Institute of Technology, Google Research), and Jia-Bin Huang (University of Maryland College Park) * Equal contributions International Conference on Computer Vision (ICCV), 2023 📝 Paper: https://arxiv.org/abs/2308.03757 🌐 Website: https://3d-motion-magnification.github.io/ 💻 Code: Coming soon! 📄 Abstract: Video motion magnification helps us visualize subtle, imperceptible motion. Prior methods, however, are only applicable to 2D videos. We present 3D motion magnification techniques that allow us to magnify subtle motions in dynamic scenes while supporting rendering from novel views. Our core idea is to represent the dynamic scene with time-varying radiance fields and leverage the Eulerian principle for motion magnification to analyze and amplify the embedding features from a fixed point over time. We study and validate the capability of 3D motion magnification for both implicit and explicit/hybrid NeRF models. We evaluate the effectiveness of our approaches on both synthetic and real-world dynamic scenes under various capture setups. Mannequin challenge videos shown in the video: - NBA: https://www.youtube.com/watch?v=5ZzklOEGW0w - MC2: https://www.youtube.com/watch?v=Z9Ag0aQYnHM - Gym: https://www.youtube.com/watch?v=wNPxoe7UBl4 - Cristiano Ronaldo: https://www.youtube.com/watch?v=n_rWD4wDSj8
DeepCamp AI