Generative Models for Discovering Sparse Distributed
Representations
Geoffrey E. Hinton and Zoubin Ghahramani
Department of Computer Science
University of Toronton
Abstract
We describe a hierarchical, generative model that can be viewed as a
non-linear generalization of factor analysis and can be implemented in a neural network.
The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual
inference correctly. Once perceptual inference has been performed the connection strengths
can be updated using a very simple learning rule that only requires locally available
information. We demonstrate that the network learns to extract sparse, distributed,
hierarchical representations.
Download [ps] [pdf]
Philosophical Transactions of the Royal Society of London,
B, 352: 1177-1190
[home page] [publications]