Early anticipation of driver's maneuver in semiautonomous vehicles using deep learning

摘要

Making machines to anticipate human action is a complex research problem. Some of the recent research studies on computer vision and assistive driving have reported that the anticipation of driver's action few seconds in advance is a challenging problem. These studies are based on the driver's head movement tracking, eye gaze tracking, and spatiotemporal interest points. The study is aimed to address an important question of how to anticipate a driver's action while driving and improve the anticipation time. The goal of this study is to review the existing deep learning framework for assistive driving. This paper differs from the existing solutions in two ways. First, it proposes a simplified framework using the driver's inside video data and develops a driver's movement tracking (DMT) algorithm. Majority of the existing state of the art is based on inside and outside features of the vehicles. Second, the proposed work tends to improve the image pattern recognition by introducing a fusion of spatiotemporal data points (STIPs) for movement tracking along with eye cuboids and then action anticipation by using deep learning. The proposed DMT algorithm tracks the driver's movement using STIPs from the input video. Also, a fast eye gaze algorithm tracks eye movements. The features extracted from STIP and eye gaze are fused and analyzed by a deep recurrent neural network to improve the prediction time, thereby giving a few extra seconds to anticipate the driver's correct action. The performance of the DMT algorithm is compared with the previous algorithms and found that DMT offers 30% improvement with regards to anticipating driver's action over two recently proposed deep learning algorithms.