Pedagogical Agent Responsive to Eye Tracking in Educational VR

摘要

We present an architecture to make a VR pedagogical agent responsive to shifts in user attention monitored by eye tracking. The behavior-based AI includes low-level sensor elements, sensor combiners that compute attention metrics for higher-level sensors called generalized hotspots, an annotation system for arranging scene elements and responses, and its response selection system. We show that the techniques can control the playback of teacher avatar clips that point out and explain objects in a VR oil rig for training.