Variational learning in nonlinear Gaussian belief
networks
Brendan J. Frey and Geoffrey E. Hinton
Department of Computer Science
University of Toronto
We view perceptual tasks such as vision and speech recognition as inference problems where
the goal is to estimate the posterior distribution over latent variables (e.g.,
depth in stereo vision) given the sensory input. The recent flurry of research in
independent component analysis exemplifies the importance of inferring the
continuous-valued latent variables of input data. The latent variables found by this
method are linearly related to the input, but perception requires nonlinear inferences
such as classification and depth estimation. In this paper, we present a unifying
framework for stochastic neural networks with nonlinear latent variables. Nonlinear units
are obtained by passing the outputs of linear Gaussian units through various
nonlinearities. We present a general variational method that maximizes a lower bound on
the likelihood of a training set and give results on two visual feature extraction
problems. We also show how the variational method can be used for pattern classification
and compare the performance of these nonlinear networks with other methods on the problem
of handwritten digit recognition.
Download [ps] [pdf]
Neural Computation 11:1, 193-214.
[home page] [publications]