Methods and systems for generating a merged reality scene based on a virtual object and a real world object represented from different vantage points in different video data streams

摘要

An exemplary merged reality scene capture system (system) is a first frame set of surface data frames from a plurality of three-dimensional (3D) capture devices arranged in relation to a real world scene to have a plurality of different advantageous points of the real world scene. Receive Based on the first set of frames, the system generates a transport stream that includes color and depth video data streams for the respective 3D capture devices. Based on the transport stream, the system generates entry description data representing a plurality of entries contained within the 3D space of the merged reality scene. The plurality of entries include virtual viewpoints in 3D space in which a second frame set of surface data frames representing color and depth data for the virtual object, real-time object, and both virtual and real-world objects are rendered.