Undesirable equilibria in systematically designed neural networks

摘要

It is possible for neural networks which have been designed using a systematic methodology to possess stable states that do not fulfil network design objectives. Two types of undesirable equilibria are described along with strategies for avoiding or escaping them. The first strategy, introducing symmetry breaking noise, is a technique which is sufficient to escape from unstable equilibria. Other techniques must be used to prevent a network from settling in undesirable but stable states. Modifying the initial state of an amplifier and reducing its gain are two techniques which are capable of enabling certain networks to escape unfeasible but stable states. If the gain of an amplifier is reduced to escape an unfeasible state, it is important to return the gain to the minimum value which will enable it to reach a digital final state. Another strategy, introducing a noise component into the update increment, has also been used successfully to escape undesirable equilibria.