GPU-Based Real-Time System for Cinematic Virtual Reality Production

摘要

Relatively inexpensive Head Mounted Display (HMD) devices made possible by advances in digital imaging technology are moving the immersive virtual and augmented reality mediums into the entertainment mainstream. This departure from the traditional art of linear story telling requires a richer production workflow and toolset capable of seamlessly integrating numerous potential narratives. An effective Virtual Reality (VR) workflow requires low-latency real-time synchronized capture of multiple live high-resolution video streams, real-time stitching and viewing of the captured assets, and real-time composition with Computer Generated (CG) elements optimally rendered for the HMD. This paper presents a real-time system for virtual reality production that encompasses optimal high-bandwidth data transfers, Graphics Processor Unit (GPU) -based video stitching, and techniques for optimally rendering to the individual display characteristics of specific VR HMDs for on-set preview.