Distributed representations were often criticized as inappropriate for en- coding of data with a complex structure. However Plate's holographic reduced representations and Kanerva's binary spatter codes are recent schemes that allow on-the-fly encoding of nested compositional struc- tures by real-valued or dense binary vectors of fixed dimensionality In this article we consider procedures of the context-dependent thin- ning developed for representation of complex hierarchical items in the architecture of associative-projective neural networks. These procedures provide binding of items represented by sparse binary codevectors (with low probability of 1s). Such an encoding is biologically plausible and allows a high storage capacity of distributed associative memory where the codevectors may be stored.
展开▼
