Perception-motivated Interpolation of Image Sequences
Abstract
We present a method for image interpolation which is able to create high-quality, perceptually convincing transitions between recorded images. By implementing concepts derived from human vision, the problem of a physically correct image interpolation is relaxed to an image interpolation that is perceived as physically correct by human observers. We find that it suffices to focus on exact edge correspondences, homogeneous regions and coherent motion to compute such solutions. In our user study we confirm the visual quality of the proposed image interpolation approach. We show how each aspect of our approach increases the perceived quality of the interpolation results, compare the results obtained by other methods and investigate the achieved quality for different types of scenes.
Publications
Correspondence Estimation and Image Interpolation for Photo-Realistic Rendering
PhD thesis, TU Braunschweig, April 2011.
Perception-motivated interpolation of image sequences
in ACM Transactions on Applied Perception, vol. 8, no. 2, pp. 1-25, February 2011.
Perception-motivated Interpolation of Image Sequences
in Proc. ACM Applied Perception in Computer Graphics and Visualization (APGV), ACM, pp. 97-106, July 2008.
http://doi.acm.org/10.1145/1394281.1394299
View and Time Interpolation in Image Space
in Computer Graphics Forum (Proc. of Pacific Graphics PG), vol. 27, no. 7, pp. 1781-1787, February 2008.
Keyframe Animation from Video
in Proc. IEEE International Conference on Image Processing (ICIP), pp. 2713-2716, October 2006.
Related Projects
Traditional optic flow algorithms rely on consecutive short-exposure images. In contrast, long-exposed images contain integrated motion information directly in form of motion blur. In this project, we use the additional information provided by a long exposure image to improve robustness and accuracy of motion field estimation. Furthermore, the long exposure image can be used to determine the moment of occlusion for the pixels in any of the short exposure images that are occluded or disoccluded.
This work has been funded by the German Science Foundation, DFG MA2555/4-1
Image-space Editing of 3D Content
The goal of this project is to develop algorithms in image space that allow photo-realistic editing of dynamic 3D scenes. Traditional 2D editing tools cannot be applied to 3D video as in addition to correspondences in time spatial correspondences are needed for consistent editing. In this project we analyze how to make use of the redundancy in multi-stereoscopic videos to compute robust and dense correspondence fields. these space-time correspondences can then be used to propagate changes applied to one frame consistently to all other frames in the video. Beside the transition of classical video editing tools we want to develop new tools specifically for 3D video content.
This project has been funded by ERC Grant #256941 `Reality CG` and the German Science Foundation, DFG MA2555/4-2.
Multi-view video camera setups record many images that capture nearly the same scene at nearly the same instant in time. Neighboring images in a multi-video setup restrict the solution space between two images: correspondences between one pair of images must be in accordance with the correspondences to the neighboring images.
The concept of accordance or consistency for correspondences between three neighboring images can be employed in the estimation of dense optical flow and in the matching of sparse features between three images.
This work has been funded in parts by the ERC Grant #256941 `Reality CG` and the German Science Foundation, DFG MA2555/4-2.
Scope of "Reality CG" is to pioneer a novel approach to modelling, editing and rendering in computer graphics. Instead of manually creating digital models of virtual worlds, Reality CG will explore new ways to achieve visual realism from the kind of approximate models that can be derived from conventional, real-world imagery as input.
The Virtual Video Camera research project is aimed to provide algorithms for rendering free-viewpoint video from asynchronous camcorder captures. We want to record our multi-video data without the need of specialized hardware or intrusive setup procedures (e.g., waving calibration patterns).