δ-NARMA Neural Networks: A Connectionist Extension of ARARMA Models

摘要

Despite their theoretical limitations, ARIMA models are widely used in real-life forecasting tasks. Parzen has proposed an extension of ARIMA models: ARARMA models. ARARMA models consist of an AR model followed by an ARMA model. Following Parzen approach, δ-NARMA neural network are MLP, the units of which are simple nonlinear ARMA-based models (∈-NARMA units). They are a non-linear extension of ARARMA models. To apply Back-Propagation Through Time algorithm to such a network, we introduce the concept of virtual error. Virtual errors can be seen as the error on hidden layer units. Such networks face the problem of non-stationary time series prediction. Experience shows that δ-NARMA networks outperform classical statistical and connectionist models on three different real-life prediction tasks. It also brings a better understanding of δ-NARMA behavior.