This directory contains a collection of R timing tests and support functions.

Written by Radford M. Neal.

This version released 2011-06-08.  Has some added tests and other
minor mods, and the example now compares R 2.11.1 with R 2.13.0
unmodified, and as modified by my speed patches of 2011-06-08.

Version of 2011-05-30 added some tests, fixes some glitches, other
minor mods.

Version of 2011-05-16 fixed some glitches, added some tests, and
improved the plot appearence.

First version was released 2011-05-12.

----------------------------------------------------------------------

Some of these test scripts are designed to test specific speed
improvement patches.  Others - eg, test-em.r, test-Qlearn.r, and
test-cholesky.r - are short programs adapted from previous real or
demo programs, intended to test general interpretive overhead.  The
tests are meant to be run on a Unix/Linux system, since Unix features
are used by the test apparatus.

The time.r file contains the support apparatus for the tests.  It
should be source before the individual test files are sourced.  Each
sequence of tests starts by setting test.name to a suitable string
(the name of the test, perhaps followed by "." and a sub-test name).
Operations are then timed with the sys.time function.  The non-trivial
parts of the operation should be in a function or functions, which
should be defined as follows:

    f <- test.cmp (function (...) {...})

The test.cmp function will either be the identity, or cmpfun, if the
function is to be compiled.

Functions are compiled if the "cmp" environment variable is set to
"T".  The "rep" environment variable controls how many times the test
operations are repeated (in order to get more accurate timing
information).  The default is 1 if this environment variable is not
set.  

The times reported are the actual times (in seconds) divided by "rep",
so that results with different numbers of repetitions are comparable.
The tests were designed so that, on the machine I was using at the
time, with unmodified R-2.13.0, they all took roughly the same time
(one second, with rep=1, uncompiled).  Of course, this will not
necessarily be the case with other machines, versions, etc.

The file do-all.r runs all the tests.  The test-*.r files can also be
run individually, once time.r has been sourced.

For example, all the tests can be run, with three repetitions, with
functions compiled, with the following shell command:

   rep=3 cmp=T R CMD BATCH do-all.r do-all.out

The results of the tests above would be stored in do-all.out.  The
times in the output are preceded by "Time:", with additional
information following that.  Most of the tests also output a few
numbers as a correctness check while they're at it, so looking at a
diff produced from two versions of R might reveal a bug.

Test results are also appended to the file "test-times", which is in a
format that can be read by read.table.  The analyse.r script contains
a command to read this file, into the variable td, and attach column
names for each item.  These columns are as follows

  date      date the test finished running
  time      time the test finished running

  c         CPU time for the test divided by repetition count
  t         name of test sequence (from test.name)
  n         number of test in sequence, from 1 up, counting from last 
            time test.name was set

  conf      version and configuration of R used, from the name of the 
            directory containing the library
  mod       modification of R, from the "mod" file in the same directory
            as the library, or "." if no such file exists

  r         run parameters, of the form "Ir" or "Cr", with "C" indicating 
            compiled or "I" interpreted, and r being the repetition count

  platform  platform specification obtained from R.version$platform

All these are printed after Time: in the output file as well, except
for date, time, and platform.

The analyse.r script also defines a function for selecting a subset of
tests, a function for giving counts of runs of different types, and
functions for calculating and plotting ratios of times.

The use of these functions is demonstrated in example-plots.r, with
the plots produced being in example.pdf. These plots show results from
tests of R version 2.11.1 and of R version 2.13.0 with and without my
speed patches released 2011-06-08.  These test results are for an
Intel X5680 processor running at 3.33GHz in 64-bit mode using gcc
4.4.4 under Red Hat Linux with default R configuration parameters.
The commands used to produce the test results are in example.sh, and
the text output of the tests is in the *.out files.  The test-times
file for the example is also included, so it should be possible to
reproduce the plots by running example-plots.r.