A Vision-Based Recognition Approach of Hand Gesture in Virtual Reality

摘要

Hand gesture capturing and recognition is an important research domain of virtual reality. However, the capturing and recognition of hand gestures is a challenging task, since the hand presents a motion of high degrees of freedom. DataGlove has been employed to capture human hand articulation by directly measuring the angles and spatial positions of the hands with sensors attached. In this paper, a vision-based method using neural network has been presented, which is cost-efficient as compared with DataGlove. A feed-forward neural network can represent an arbitrary functional mapping so it is possible to map raw data directly to the required results. We use neural networks to recognize the patterns that exist in the raw data set of the human hand gestures. In the paper, a pattern recognition system of hand gestures based on neural network has been implemented that converts an image of the hand gesture into a feature vector, which will then be compared with the feature vectors of a training set of gestures. The final system has been implemented with a three-layer neural network. The recognition system demonstrates that neural networks can be used to develop the complex mappings required in a high dimensional data. The system can be effectively used in virtual reality to perform various tasks like the manipulation of the virtual objects in virtual environment, and allow to manipulate the virtual objects in a more fast, precise, and natural way. The approach presented in this paper is also easy to extend to other similar domains.