Real-time 3D Reconstruction to Extend Game-based Virtual Reality Used in Training

摘要

Training based on virtual reality is often referred to as virtual training. It is an alternative to traditional physical training. It has the advantages with respect to high-level integration of multiple training methods (nearly all traditional training methods can be realized in the virtual world), high flexibility and convenience (no strict constraints of time and location if Internet service is available), economy (no requirement of space and materials, no need for real people), safety and reliability (necessary in dangerous circumstances and no risk of malfunctioning of virtual devices).;Although virtual training seems attractive, several disadvantages undermine its development. Firstly, the deployment of virtual training highly depends on the research of computer vision, computer graphics, artificial intelligence, dynamics, sensors, smart controllers, remote communication techniques and so on. Secondly, the feel of immersion coming from humans' five senses is insufficient. Thirdly, virtualization of the real world in order to create the virtual environments for virtual training is complicated and time consuming. In order to facilitate the development of virtual training systems, the short-comings of virtual training are discussed and then, the corresponding solutions are given.;The solution for increasing the feel of immersion is to augment VR by integrating physical devices into virtual reality. The contribution of this solution is to make virtual reality extendable and cross-platform operable using dynamic link libraries and shared memory. The solution to improve the efficiency of the creation of virtual environments is to employ real-time 3D reconstruction. The contribution of this solution is to give an algorithm used to estimate the pose of a Kinect and to fuse data while enabling the recognition for scanned objects. In addition, in order to improve the quality of the reconstruction, a new digital filter that reduces the periodic and quasi-periodic noise in the acquired data is developed.