Neural Reconstruction and Rendering of Dynamic Real-World Scenes

Abstract

The photorealistic reconstruction and representation of real-world scenes has always been an integral field of research in computer graphics, and includes traditional rendering as well as interdisciplinary techniques from computer vision and machine learning. In addition to conventional applications in photogrammetry, detailed reconstructions from camera or smartphone images have recently also enabled the automated integration of real, photorealistic content in multimedia applications such as virtual reality.

A large number of current methods focus on the 3D representation of static content. In practice, however, many scenes are subject to temporal deformation and therefore require an additional reconstruction of the temporal dimension. At the ICG, we develop technologies for the reconstruction and visualization of dynamic scenes from monocular video recordings. The methods we have developed allow not only the real-time display of new, high-resolution camera views but also the and manipulation of temporal sequences, such as the “bullet time” effect the “bullet time” effect known from Matrix. In the future, the resulting resulting models will enable exciting new applications, such as the immersive reproduction of experiences in virtual reality.