The performance of large neural networks can be judged not only by their storage capacity but also by the time required for learning. A polynomial learning algorithm with learning time proportional to N-2 in a network with N units might be practical whereas a learning time proportional to exp N would allow rather small networks only. The question of absolute storage capacity alpha(c) and capacity for polynomial learning rules alpha(p) is discussed for several feedforward architectures, the perception, the binary perceptron, the committee machine and a perceptron with fixed weights in the first layer and adaptive weights in the second layer. The analysis is based partially on dynamic mean-field theory which is valid for N --> infinity. In particular, for the committee machine a value alpha(p) considerably lower than the capacity predicted by replica theory or simulations is found. This discrepancy is resolved by new simulations investigating the learning time dependence and revealing subtleties in the definition of the capacity. [References: 14]
展开▼
