About the Benchmarks

The benchmark page shows some benchmark results on a collection of fairly standard (mostly Gabriel) Scheme benchmarks.

Tables show relative performance, with the actual time for the fastest run shown on the left. So, by default, 1 is the fastest, but select any implementation to normalize the table with respect to that implementation's speed. A - appears when a benchmark didn't run in an implementation for some reason (possibly not a good one).

Small gray numbers are (relative) compile times, where the compile time for the nothing benchmark is subtracted from every other benchmark's compile time.

Run times are averaged over three runs. All reported times are CPU time (system plus user). The times are based on the output of the implementation's time syntactic form for function.

Machine:

Compiler configurations:

These configurations are all “safe mode,” but they allow the compiler to assume that built-in Scheme functions are not redefined and that no top-level defintion is ever changed. Such assumptions correspond to putting the benchmark in an R6RS library.

In general, we attempt to use the various implementations in a compentent way, but not in a sophisticated way. For example, we do not tweak inlining parameters or specify fixnum arithmetic (where appropriate), which could produce significant improvements from some compilers.

For more benchmarks and a more sophisticated use of a few compilers, including fixnum- and flonum-specific arithmetic as well as unsafe modes, see this other page.

For further details on the benchmarks here, see the benchmark source and infrastructure, which is available form the PLT SVN repository:

http://svn.plt-scheme.org/plt/trunk/collects/tests/mzscheme/benchmarks/common/