Does the Wake-Sleep Algorithm Produce Good Density
Brendan J. Frey, Geoffrey E. Hinton
and Peter Dayan
Department of Computer Science
University of Toronto
The wake-sleep algorithm (Hinton, Dayan, Frey
and Neal 1995) is a relatively efficient method of fitting a multilayer stochastic
generative model to high-dimensional data. In addition to the top-down connections in the
generative model, it makes use of bottom-up connections for approximating the probability
distribution over the hidden units given the data, and it trains these bottom-up
connections using a simple delta rule. We use a variety of synthetic and real data sets to
compare the performance of the wake-sleep algorithm with Monte Carlo and mean field
methods for fitting the same generative model and also compare it with other models that
are less powerful but easier to fit.
Download [ps] [ps.gz] [pdf]
In Advances in Neural Information Processing Systems 8. MIT
Press (1996): Cambridge, MA.
Presented at the Neural Information Processing Systems Conference, Denver,
Colorado, Dec. 1995.
[home page] [publications]