NOTES ON THE VERSION OF 2000-08-21 Major features of this release: o Simulations can (sometimes) be done using "circular coupling", as described in my technical report on "Circularly-coupled Markov chain sampling". Implementing this required a number of changes and additions to the software. o New multivariate slice sampling procedures have been implemented, described in my technical report on "Slice sampling" (a revision of the earlier report on "Markov chain Monte Carlo methods based on `slicing' the density function"). o A new example has been added of sampling from a "funnel" distribution, which demonstrates some problems with MCMC, and how slice sampling can help overcome these problems. Various other less major changes have also been made. Changes in this version: 1) A new xxx-circ command has been added (see xxx-circ.doc), which does a circularly-coupled simulation to produce a wrapped-around chain. Multiple starting points are used in order to diagnose convergence problems. The simulation can be done sequentially or in parallel. 2) A new xxx-wrap command has been added (see xxx-wrap.doc), which does simple simple circularly-coupled simulations with just one starting point. It produces a wrapped-around chain starting from an existing log file for a standard simulation. 3) Simple examples of circular coupling done using dist-wrap and dist-circ have been added. See Ex-circ.doc. 4) When acceptance decisions for Metropolis, hybrid Monte Carlo, and other Markov chain operations are made, the random number sometimes required is now ALWAYS generated, so as to keep the random number stream synchronized for two runs with the same random seed but different starting points. Previously, this random number was not generated if the energy change was negative, since acceptance was then guaranteed regardless of the random number chosen, but this undermines coupling. Note that some Markov chain operations are still not appropriate for use in circularly-coupled simulations, including slice sampling operations and updates for second-level neural network hyperparameters. As a result of this change, redoing previous runs with the new version will typically produce different results, but the distribution sampled should be the same. 5) New "rgrid-met" and "rgrid-met-1" operations have been added, designed to produce coalescence of coupled chains once they are close together. See mc-spec.doc for details. 6) A new xxx-genp program has been written that generates random momentum variables (see xxx-genp.doc). Such a program is not normally required, since momentum variables are generated automatically by xxx-mc if they don't exist and are needed, but explicit generation is needed to ensure random number synchronization in xxx-circ. 7) A new log-equal program has been added that tests whether or not the records stored at given indexes in given log files match exactly. This was needed to write xxx-circ. See log-equal.doc for details. 8) An option has been added to allow the indexes of records appended with log-append to start with a specified index or the last existing index, rather than one past the last index. This was needed to write xxx-circ. See log-append.doc for details. 9) Options have been added to log-copy and log-append to allow them to ignore records of certain types. This is for consistency with the new log-equal command, and because it's needed for xxx-wrap. For details, see log-append.doc and log-copy.doc. 10) A "-b" option has been implemented for all the Metropolis updates, which causes acceptance decisions to be made with the "Barker" / "Boltzmann" function, rather than the usual Metropolis function. This is so I could see if this makes a difference to coalescence speed when coupling. See mc-spec.doc for details. 11) A "-r" option has been implemented for all single-variable Metropolis and slice sampling updates, which causes just one randomly-chosen variable to be updated each iteration, rather than all of them. See mc-spec.doc for details. 12) When doing single-variable slice sampling with slice-1 or slice-over with max-steps set to 1, the random number for allocating steps to the left or right is no longer generated, as it has no effect when max-steps is 1. This will cause results of runs to differ in detail from before, due to the subsequent random number sequence differing. 13) A "slice" and "slice-gaussian" operations that do multidimensional slice sampling have been added. See mc-spec.doc and my new tech report on "Slice sampling". 14) A new example of sampling from a "funnel" distribute has been added, showing an interesting/disturbing problem, and how slice sampling can help avoid it. 15) The 'dist' module now allows specification of Bayesian models in which the entire likelihood function is given as a formula, with the data appearing in the formula as constants. This is an alternative to the normal method of giving a formula for the likelihood due to one case, with the data for that case filled in from a data file. See dist-spec.doc for details. 16) The defaults for the get2 and opt2 trajectory computation methods have been changed to "firstp" - ie, update the momentum first. This is for consistence with the leapfrog method, and because this is sometimes better than "firstq". 17) The number of iterations in the examples of sampling from the "ring" distribution in Ex-dist-g.doc has been increased to 2000, so that autocorrelations will be better estimated. The new multivariate slice sampling operations are now included. Bug fixes. 1) Fixed a bug in dist-est that caused the estimate for the standard error of the mean to be wrong if the number of data points exceeded 46340. 2) Fixed a bug that affected use of trajectory computation methods other than the standard "leapfrog" for neural network models. 3) The 'formula' module, used by the 'dist' module, has been altered to allow it to compute the gradient of a*b and a/b when b is undefined but a and its gradient are both zero. Known bugs and other deficiencies. 1) The facility for plotting quantities using "plot" operations in xxx-mc doesn't always work for the first run of xxx-mc (before any iterations exist in the log file). A work-around is to do a run of xxx-mc to produce just one iteration before attempting a run of xxx-mc that does any "plot" operations. 2) The CPU time features (eg, the "k" quantity) will not work correctly if a single iteration takes more than about 71 minutes. 3) The latent value update operations for Gaussian processes may recompute the inverse covariance matrix even when an up-to-date version was computed for the previous Monte Carlo operation. 4) Covariance matrices are stored in full, even though they are symmetric, which sometimes costs a factor of two in memory usage. 5) The "p" option of net-pred isn't allowed for survival models with piecewise-constant hazard. There's no big reason for this - I just haven't gotten around to it.