NOTES ON THE VERSION OF 1997-07-22:

Changes in this version:

1) Programs for finite and infinite mixture models have been added.
   The facilities provided are currently somewhat minimal, but are
   adequate for doing some interesting things.  Two new examples
   demonstrating the mixture models have been added.

2) A program for training networks by old-fashioned gradient descent
   has been added, to allow experimental comparisons.  See net-gd.doc
   for details.  A program useful for doing early stopping using
   cross-validation has also been added (see find-min.doc).

3) The exponential parts of a Gaussian process specification can now
   include "delta" and "omit" options.  The delta option is useful
   for categorical inputs.  The omit option allows one to explicitly
   set up additive models.  See gp-spec.doc for details.

4) One can now specify that the noise in a Gaussian process regression 
   model is autocorrelated (for training cases).  See model-spec.doc
   for details.

5) A new "M" quantity has been provided for Gaussian process models,
   which summarizes the magnitude of an input's relevance.  See 
   gp-quantities.doc for details.

6) A log-append program has been added, which allows a run to be 
   initialized with a state from another run, provided the other
   run is compatible with the new in all required respects.

7) A gp-cov program has been added that prints the covariance matrix 
   for a Gaussian process.  A gp-eigen program has also been added, 
   which computes the eigenvalues and/or eigenvectors of this 
   covariance matrix.

8) Routines have been added to util/matrix.c for initializing an 
   identity matrix and finding eigenvalues and eigenvectors by Jacobi
   iteration.

9) The output of net-pred and gp-pred when the 'b' or 'B' option
   is used has been changed so that the numbers are printed to high
   precision, in exponential format.

10)The accuracy of the Monte Carlo procedure for making predictions
   for binary targets with Gaussian process models has been improved
   somewhat, using post-stratification.

11)The stepsize heuristics for Gaussian processes have been changed
   slightly.  The stepsizes for the noise hyperparameters are now 
   half what they were, which seems to be a closer fit to the
   typical situation.  It may now be possible to increase the stepsize
   adjustment factor used, and thereby obtain more efficient exploration 
   of the hyperparameter space.

12)The stepsize heuristics for neural networks with more than one 
   hidden layer have also been changed, by fixing a bug.  The hope
   is that this change will also be an improvement, without the need
   for a change in the stepsize adjustment factor used, but this 
   might not always be true.

Bug fixes: Fixed a bug that made the "radial heatbath" Markov chain
method sometimes not work.  Fixed a bug in net-eval that affected it
when the "targets" option was used.  Fixed the bug in network stepsizes
mentioned in (10) above.  Some bugs in error checking fixed.

Portability: The modes for "fopen" have been changed, since the old
ones didn't work on some systems.  Some other changes that seem
advisable but which probably weren't actually causing problems have
been made as well.

As far as I know, log files created with the previous versions back
to that of 1996-08-26 should still be readable with the new version, 
on the same machine as they were created on.


Known bugs and other problems.

1) The facility for plotting quantities using "plot" operations in xxx-mc
   doesn't always work for the first run of xxx-mc (before any
   iterations exist in the log file).  A work-around is to do a run of
   xxx-mc to produce just one iteration before attempting a run of
   xxx-mc that does any "plot" operations.

2) The CPU time features (eg, the "k" quantity) will not work correctly
   if a single iteration takes more than about 71 minutes.

3) The "sample-values" and "scan-values" operations for Gaussian processes
   always recompute the inverse covariance matrix, even when an up-to-date
   version was computed for the previous Monte Carlo operation.