Probabilistic Pose Estimation and 3D Reconstruction of Vehicles from Stereo Images

摘要

The pose estimation and reconstruction of 3D objects from images is one of the major problems that are addressed in computer vision and photogrammetry. The understanding of a 3D scene and the 3D reconstruction of specific objects are prerequisites for many highly relevant applications of computer vision such as mobile robotics and autonomous driving. To deal with the inverse problem of reconstructing 3D objects from their 2D projections, a common strategy is to incorporate prior object knowledge into the reconstruction approach by establishing a 3D model and aligning it to the 2D image plane. However, current approaches are limited due to inadequate shape priors and the insufficiency of the derived image observations for a reliable association and alignment with the 3D model. The goal of this thesis is to infer valuable observations from the images and to show how 3D object reconstruction can profit from a more sophisticated shape prior and from a combined incorporation of the different observation types. To achieve this goal, this thesis presents three major contributions for the particular task of 3D vehicle reconstruction from street-level stereo images. First, a subcategory-aware deformable vehicle model is introduced that makes use of a prediction of the vehicle type for a more appropriate regularisation of the vehicle shape. Second, a Convolutional Neural Network (CNN) is proposed which extracts observations from an image. In particular, the CNN is used to derive a prediction of the vehicle orientation and type, which are introduced as prior information for model fitting. Furthermore, the CNN extracts vehicle keypoints and wireframes, which are well-suited for model association and model fitting. Third, the task of pose estimation and reconstruction is addressed by a versatile probabilistic model. Suitable parametrisations and formulations of likelihood and prior terms are introduced for a joint consideration of the derived observations and prior information in the probabilistic objective function. As the objective function is non-convex and discontinuous, a proper customized strategy based on stochastic sampling is proposed for inference, yielding convincing results for the estimated poses and shapes of the vehicles. To evaluate the performance and to investigate the strengths and limitations of the proposed method, extensive experiments are conducted using two challenging real-world data sets: the publicly available KITTI benchmark and the ICSENS data set, which was created in the scope of this thesis. On both data sets, the benefit of the developed shape prior and of each of the individual components of the probabilistic model can be shown. The proposed method yields vehicle pose estimates with a median error of up to 27 cm for the position and up to 1.7° for the orientation on the data sets. A comparison to state-of-the-art methods for vehicle pose estimation shows that the proposed approach performs on par or better, confirming the suitability of the developed model and inference procedure.