Robust spatiotemporal analysis of architectural imagery.

摘要

This thesis addresses the issue of understanding and manipulating images of architectural scenes. Automatically modeling the structure and appearance of buildings with a robot is challenging; an end-to-end system would have to tackle a whole spectrum of tasks such as planning, sensor fusion, navigation, image acquisition and matching, structure estimation and texture mapping. Purely bottom-up techniques are inadequate for this task due to ambiguities and missing information inherent in sensor data. My solution is to introduce additional domain-specific models that can capture dependencies such as restricted spatial configurations or geometric patterns in images of buildings. Techniques to encode, discover and exploit these relationships for retrieving semantic information about buildings are illustrated. These interaction models are shown to be powerful for such varied tasks as object recognition and detection, segmentation, inference of missing information, and realistic image synthesis---even without supervised training or other appearance models.; A primary focus of this work is on constructing "clean'' texture map mosaics of building facades. Without explicit handling, foreground objects such as trees, signs, and people will appear pasted as artifacts on the model. As a first major contribution, given an image sequence captured around the building, I developed a novel spatiotemporal timeline-based inpainting technique to remove non-building pixels from the median mosaic. These polluted regions are a result of the majority of views being occluded, which makes conventional techniques such as the median filter unreliable. Outlier pixels are then automatically identified by a robust measure of spread. A combination of motion cues and an automatically trained appearance-based classifier are used to fill the majority occluded holes with true building background. A second stage of spatial inpainting is applied to the relatively small unanimously occluded regions in which the background was never imaged. Results are shown on a variety of campus buildings.; My second major innovation is a series of methods that enable foreground removal from single images of buildings or brick walls without any motion information. The key insight is to use a priori knowledge about grid patterns on building facades that can be modeled as Near Regular Textures (NRT). I describe a Markov Random Field (MRF) model for such textures and introduce a Markov Chain Monte Carlo (MCMC) optimization procedure for discovering grid structures on building images. Results are shown on both synthetic NRT as well as building images. This simple spatial rule is then used as a starting point for inference of missing windows, facade segmentation, grammar-based image parsing, outlier identification, and foreground removal.; I also describe related work on how aerial imagery may be exploited for navigating a robot around the building perimeter. A randomized approach to view planning is presented that generates paths to simultaneously address visual coverage and quality. A Monte Carlo Localization framework for vehicle localization and guidance is also described.