Geometry and Expressive Power of Conditional Restricted Boltzmann Machines

摘要

Conditional restricted Boltzmann machines are undirectedstochastic neural networks with a layer of input and outputunits connected bipartitely to a layer of hidden units. Thesenetworks define models of conditional probability distributionson the states of the output units given the states of the inputunits, parameterized by interaction weights and biases. Weaddress the representational power of these models, provingresults on their ability to represent conditional Markov randomfields and conditional distributions with restricted supports,the minimal size of universal approximators, the maximal modelapproximation errors, and on the dimension of the set ofrepresentable conditional distributions. We contribute new toolsfor investigating conditional probability models, which allow usto improve the results that can be derived from existing work onrestricted Boltzmann machine probability models. color="gray">