Artificial ``neural networks'' are now widely used as flexible models
for regression and classification applications, but questions remain
regarding what these models mean, and how they can safely be used when
training data is limited. *Bayesian Learning for Neural
Networks* shows that Bayesian methods allow complex neural network
models to be used without fear of the ``overfitting'' that can occur
with traditional neural network learning methods. Insight into the
nature of these complex Bayesian models is provided by a theoretical
investigation of the priors over functions that underlie them. Use of
these models in practice is made possible using Markov chain Monte
Carlo techniques. Both the theoretical and computational aspects of
this work are of wider statistical interest, as they contribute to a
better understanding of how Bayesian methods can be applied to complex
problems.

Presupposing only basic knowledge of probability and statistics, this book should be of interest to many researchers in Statistics, Engineering, and Artificial Intelligence. Software for Unix systems that implements the methods described is freely available over the Internet.

Lecture Notes in Statistics No. 118, Springer-Verlag New York, 1996, ISBN 0-387-94724-8, free download from Springer site.

The neural network programs that go with the book are now part of my software for flexible Bayesian modeling.

Neal, R. M. (1994)Chapter 2 ofBayesian Learning for Neural Networks, Ph.D. Thesis, Dept. of Computer Science, University of Toronto, 195 pages: abstract, postscript, pdf, associated references, associated software.

Neal, R. M. (1994) ``Priors for infinite networks'', Technical Report CRG-TR-94-1, Dept. of Computer Science, University of Toronto, 22 pages: abstract, postscript, pdf.Chapter 3 is a further development of ideas in the following papers:

Neal, R. M. (1993) ``Bayesian learning via stochastic dynamics'', in C. L. Giles, S. J. Hanson, and J. D. Cowan (editors)Advances in Neural Information Processing Systems 5, pp. 475-482, San Mateo, California: Morgan Kaufmann: abstract.Neal, R. M. (1992) ``Bayesian training of backpropagation networks by the hybrid Monte Carlo method'', Technical Report CRG-TR-92-1, Dept. of Computer Science, University of Toronto, 21 pages: abstract, postscript, pdf.