home page
people
research
publications
seminars
travel
search
UCL

The wake-sleep algorithm for unsupervised neural networks

Geoffrey E. Hinton, Peter Dayan,
Brendan J. Frey and Radford M. Neal
Department of Computer Science
University of Toronto

Abstract

We describe an unsupervised learning algorithm for a multilayer network of stochastic neurons. Bottom-up 'recognition' connections convert the input into representations in successive hidden layers and top-down 'generative' connections reconstruct the representation in one layer from the representation in the layer above. In the 'wake' phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the 'sleep' phase, neurons are driven by generative connections and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.

Download  [ps]  [pdf]

Science, 268, 1158-1161. (1995)

[home page]  [publications]