Image- and Video-based Rendering

The purpose of image- and video-based rendering (IVBR) is to be able to synthesize photo-realistic new, virtual views of real-world scenes and events from no more than a set of conventional photographs or videos of the scene. Purely image-based, or plenoptic, approaches densely sample scene appearance using a large number of images. New views are generated by simply re-sampling the captured image data. In contrast, geometry-assisted methods require much less input images. Here, 3D scene geometry is either a-priori known, reconstructed from the captured imagery, or acquired separately by some other means, e.g., ranging

imaging. With (approximate) scene geometry available, views from arbitrary viewpoints are synthesized using the acquired images as geometry texture. Over the years, various IVBR methods have been proposed that can all be categorized in-between these two limiting cases.

This chapter gives an overview of the most seminal techniques.