Script for mlp-mc-1.

The script outlined below is used in conjunction with Radford Neal's software package to implement the mlp-mc-1 method.

  net-spec log.$1 8 6 1 / - x0.2:0.5:1 0.1:0.5 - x0.05:0.5 x0.2:0.5:1 1
  model-spec log.$1 real 0.05:0.5
  data-spec log.$1 8 1 / train.$1 .

  rand-seed log.$1 $1
  net-gen log.$1 fix 0.1 0.5
  mc-spec log.$1 repeat 10 sample-noise heatbath hybrid 100:10 0.15
  net-mc log.$1 1
  mc-spec log.$1 sample-sigmas heatbath 0.95 hybrid 100:10 0.15 negate
  net-mc log.$1 @32 58

  net-pred bn log.$1 @10.7:32+100 / test.$1 > cguess.S.$1
  net-pred bd log.$1 @10.7:32+100 / test.$1 > cguess.A.$1
  net-pred bp log.$1 @10.7:32+100 / test.$1 targets.$1 > clptarg.L.$1

The argument to the script $1 contains the instance number. In this example a network with 8 inputs and a single layer of 6 hidden units is trained for 32 minutes of cpu-time. The number of hidden units should be selected according to the following heuristics: Use the smallest number of hidden units such that the total number of weights is at least as large as the number of training examples with a lower limit of 6 hidden units and an upper limit of 25. The net-spec command specifies the architecture of the network and the weight priors. The following lines specify the noise model and tells the program where to get the data files. After setting the seed for the random number generator the hyperparameters are initialized. Then the Markov chain is set up to perform 10 iterations each of 100 leapfrog steps with a step size of 0.15 for fixed values of the hyperparameters. These iterations are performed in the following call to net-mc and should allow the weights to attain somewhat reasonable values.

In the following line the Markov chain is set up for the remainder of the run. The sample-sigmas command updates the hyperparameters and the heatbath 0.95 command updates momenta with a persistance of 0.95; the remainder of the line specifies Hybrid Monte Carlo for the weights with leaprog trajectories of length 100, window size 10 and step size 0.15. The call to net-mc allows for 32 minutes of cpu time and saves every 58th iteration to the log file - an interval which by experimentation was determined appropriate for the desired number of samples in 32 minutes. The last three lines of the script generate predictions from 100 samples from the last 2/3 of the run for the three standard loss types.

Last Updated by Carl Edward Rasmussen, September 13, 1996