Parallax Panorama Video
Abstract
Recent advances in consumer-grade panorama capturing setups enable personalized 360° experiences. Along with the improvement in head-mounted displays (HMDs), these allow to bring back memories at a previously unprecedented level of immersion. However, the lack of explicit depth and scene geometry prohibits any form of head movement, which is needed for a fully-immersive VR experience.
This research project explores methods for adding motion parallax to previously captured stereo panorama videos and enabling real-time playback of these enhanced videos in HMDs.
Funding
This work is being funded by the German Science Foundation (DFG) under the Reinhart Koselleck Project "Immersive Digital Reality" (DFG MA2555/15-1).
Publications
Immersive Free-Viewpoint Panorama Rendering from Omnidirectional Stereo Video
in Computer Graphics Forum, vol. 42, no. 6, John Wiley & Sons, Inc., pp. e14796 ff., April 2023.
Temporal Consistent Motion Parallax for Omnidirectional Stereo Panorama Video
in ACM Symposium on Virtual Reality Software and Technology (VRST), no. 21, Association for Computing Machinery, pp. 1-9, November 2020.
Depth Augmented Omnidirectional Stereo for 6-DoF VR Photography
in Proc. IEEE Virtual Reality (VR) Workshop, IEEE, pp. 660-661, May 2020.
Multiview Panorama Alignment and Optical Flow Refinement
in Magnor M., Sorkine-Hornung A. (Eds.): Real VR – Immersive Digital Reality: How to Import the Real World into Head-Mounted Immersive Displays, Springer International Publishing, Cham, ISBN 978-3-030-41816-8, pp. 96-108, March 2020.
Related Projects
Eye-tracking Head-mounted Display
Immersion is the ultimate goal of head-mounted displays (HMD) for Virtual Reality (VR) in order to produce a convincing user experience. Two important aspects in this context are motion sickness, often due to imprecise calibration, and the integration of a reliable eye tracking. We propose an affordable hard- and software solution for drift-free eye-tracking and user-friendly lens calibration within an HMD. The use of dichroic mirrors leads to a lean design that provides the full field-of-view (FOV) while using commodity cameras for eye tracking.
Real VR - Immersive Digital Reality
With the advent of consumer-market Virtual Reality (VR) technology, the next revolution in visual entertainment is already on the horizon: real VR will enable us to experience live-action movies, sports broadcasts, concert videos, etc. in true visual (and aural) immersion. This book provides a comprehensive overview of the algorithms and methods that make it possible to immerse into real-world recordings. It brings together the expertise of internationally renowned experts from academia and industry who present the state of the art in this fascinating, interdisciplinary new research field. Written by and for scientists, engineers, and practitioners, this book is the definitive reference for anyone interested in finding out about how to import the real world into head-mounted displays.