Web3D Client-Enhanced Global Illumination via GAN for Health Visualization

摘要

3D visualization of digital human becomes a key tool for the medical visualization, especially for medical education. Web3D technology has been commonly applied in this field. However, the quality of rendering is not expected for the medical purpose. Nowadays, global illumination (GI) map is an efficient tool for real-time lighting and shadow rendering. On the cloud baking server, a large number of rendered GI maps are generated under variety of configuration in the scene on the Web3D interface end. GI tree works on organizing these baked maps for reusing in the Web3D client. Meanwhile, it dispatches the existing baked maps directly in the case that the viewpoint appears in the duplicate positions in the Web3D client. This is the main stream solution of the cloud pre-rendering. However, it is a challenge to store and manage excessive rendered maps. In this paper, we propose a light-weight collaborative machine learning method for lighting and shadow rendering in medical applications. In this system, the conditional generative adversarial networks (GAN) works for generating the GI map instead of finding out the similar from number of stored maps, and we propose structure-aware 3D image warping method to improve the system performance. The experiments demonstrated that our proposed system not only guarantees the resolution of the GI map in the Web3D client, but also significantly reduces the rendering computational needs so as to improve the system performance.