Immersive 3D Video is Coming
About this lesson
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhöfer, and Johannes Kopf, Matthew O'Toole, and Changil Kim IEEE/CFV Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight ⭐⭐⭐) 📝 Paper: https://arxiv.org/abs/2301.02238 🌐 Website: https://hyperreel.github.io/ 💻 Code: https://github.com/facebookresearch/hyperreel 📄 Abstract: Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques. However, the volume rendering procedures that drive these representations necessitate careful trade-offs in terms of quality, rendering speed, and memory efficiency. In particular, existing methods fail to simultaneously achieve real-time performance, small memory footprint, and high-quality rendering for challenging real-world scenes. To address these issues, we present HyperReel — a novel 6-DoF video representation. The two core components of HyperReel are: (1) a ray-conditioned sample prediction network that enables high-fidelity, high frame rate rendering at high resolutions and (2) a compact and memory-efficient dynamic volume representation. Our 6-DoF video pipeline achieves the best performance compared to prior and contemporary approaches in terms of visual quality with small memory requirements, while also rendering at up to 18 frames-per-second at megapixel resolution without any custom CUDA code.
DeepCamp AI