Human action recognition toward massive-scale sport sceneries based on deep multi-model feature fusion

摘要

In sport sceneries, automatically recognizing human actions is a useful technique that can be popularly applied in may domains, such as human body tracking and athlete behavior analysis Most state-of-the-art deep architectures have achieved competitive performance in recognizing human action. However, it is still a challenging task due to the unavoidable occlusion, camera angle changes, and varied human posture. In this paper, we propose a novel deep multimodal feature fusion algorithm for human action recognition. The key technique is a multi-model feature fusion scheme. More specifically, we fuse visual feature, skeleton posture, probability maps and audio signal into a hybrid feature, which is utilized to represent human action. Then these feature channels are optimally combined using a deep model, wherein the weights of multiple feature channels can be predicted intelligently. Finally, the optimally fused feature are fed into a multi-class SVM for conducting human action recognition. Extensive comparative results and parameter analysis have shown the effectiveness of our proposed method.