Learning method for multilayer perceptron neural network with N- bit data representation

摘要

A multilayer perceptron neural network with N-bit (8-bit) data representation generates weighted sums in forward and backward calculations having 2N-bit data precision. During N-bit digital learning of a multilayer perceptron, the maximum value represented with N-bits is set to a value corresponding to the sigmoidal saturated region when the result of the weighted sum having 2N-bit data precision in the forward calculation of the multilayer perceptron is represented with N-bit data for a sigmoidal nonlinear transformation. The maximum value of N-bit presentation is set to a value comparatively smaller than that represented in 2N bits when the result of the weighted sum having the 2N- bit data precision in the backward calculation of the multilayer perceptron is represented with N-bit data. With the representation range of the weights being small, if a predetermined ratio of the weights approaches a maximum value according to the learning progress, the weight range is expanded. The efficiency of 8-bit digital learning can be enhanced to approach that of 16-bit digital learning.