AN IMPROVED DIAGONAL RECURRENT NEURAL NETWORKS ITERATIVE ALGORITHM

摘要

A new improved iterative algorithm for diagonal recurrent neural networks (DRNN) is presented in this paper. DRNN used is a three layers network, which has only one hidden layer with sigmoid recurrent neurons and has three kinds of weights, input weight matrix, recurrent weight vector and output weight vector. The input weight matrix is composed of all connecting weights from all inputs of DRNN and all recurrent neurons. The recurrent weight vector is composed of connecting weight from every recurrent neuron to itself. The output weight vector is composed of all connecting weights from recurrent neurons to output neuron. To get much fast convergence speed, in this paper, input weight matrix between all inputs and all hidden neurons are divided into many input weight vectors composed of weights between all inputs and a certain hidden neuron. To guarantee convergence, Learning rate of every input weight vector is formulated by introducing a Lyapunov function. Then, learning rate of input weight matrix is chosen as the smallest one among all the input weight vectors' learning rates. A DRNN control system is presented for numerical simulation. It is shown that, the learning rate got in this paper is higher than those in past literatures, and the convergence speed of DRNN is much faster, also. Numerical simulation results are provided to confirm the performance and effectiveness of the proposed method.