Omnidirectional Multicamera Video Stitching Using Depth Maps

摘要

Omnidirectional vision has recently captured plenty of attention within the computer vision community. The popularity of cameras able to capture 360 degrees has increased in the last few years. A significant number of these cameras are composed of multiple individual cameras that capture images or videos, which are stitched together at a later postprocess stage. Stitching strategies have the complex objective of seamlessly joining the images, so that the viewer has the feeling the panorama was captured from a single location. Conventional approaches either assume that the world is a simple sphere around the camera, which leads to visible misalignments on the final panoramas, or use feature-based stitching techniques that do not exploit the rigidity of multicamera systems. In this paper, we propose a new stitching pipeline based on state-of-the-art techniques for both online and offline applications. The goal is to stitch the images taking profit of the available information on the multicamera system and the environment. Exploiting the spatial information of the scene helps to achieve significantly better results. While for the online case, sparse data can be obtained from a simultaneous localization and mapping process, for the offline case, it is estimated from a 3-D reconstruction of the scene. The information available is represented in depth maps, which provide all information in a condensed form and allow easy representation of complex shapes. The new pipelines proposed for both online and offline cases are compared, visually and numerically, against conventional approaches, using a real data set. The data set was collected in a challenging underwater scene with a custom-designed multicamera system. The results obtained surpass those of conventional approaches.