A hierarchical LSTM model with attention for modeling EEG non-stationarity for human decision prediction

摘要

Recent progress in using Long Short-Term Memory (LSTM) Network in sequence-to-sequence learning of video, text, image has motivated us to explore its use Electroencephalogram (EEG) sequence signals. However, modeling an EEG sequence is a challenging task due to its high dimensionality and non-stationarity. The goal of this work is to predict the human decision from continuous EEG signals. An application was designed to guard a restricted area, a decision of allow or deny is made based on the physical appearance and identification card. In this paper, we propose a hierarchical long short-term memory (H-LSTM) model with attention, where the first layer encodes local-temporal correlations between EEG time-samples in local epochs and the second layer encodes the temporal correlations between epochs in a sequence. Thus, this novel model can address non-stationarities in EEG data. The proposed model highlights the time points contributing to classification of human decision made at an epoch level. Classification results of guard's decision (Allow/Deny) is reported from 18 participants. Our results indicate that H-LSTM model outperforms an LSTM model by 12.4% and a shallow Support Vector Machine model by 17.4% Our results suggest that the H-LSTM model can be utilized effectively to predict human decision or other similar applications.