This commit does four things:
* Adds "pb.ss" and "pb.c", which implement a portable bytecode
backend and interpreter that is intended for bootstrapping. A
single set of pb bootfiles can support bootstrapping on all
platforms --- as long as the C compiler supports a 64-bit integer
type. The pb machine supports foreign calls for only a small set of
recognized prototypes, and it does not support foriegn callables.
Use `./configure --pb` to build the pb variant.
* Changes the kernel's casts between `ptr` and `void*` types. In a pb
build, the `ptr` type can be a 64-bit integer type while `void*` is
a 32-bit pointer type, so casts must go through an intermediate
integer type.
* Adjusts the compiler to accomodate run-time-determined endianness.
Making the compiler agnostic to word size is not practical, but
only a few pieces depend on the target machine's endianness, and
those can generally be deferred to a run-time choice of byte-based
operations. The one exception is that ftype bit fields are not
allowed unless accompanied by an explicit endianness declaration.
* Start reducing duplication among platform-specific makefiles. For
example, `Mf-ta6osx` chains to `Mf-a6osx` to avoid repeating most
of it. A lot more can be done here.
original commit: 97533fa9d8b8400b0dc1a890768c7d30c91257e0
Merge changes in the way that fasl streams are compressed. The new
approach makes compression explicit in the fasl representation, which
means that tricks like uzing zcat on a fasl file will no longer work
(at least not efficiently).
original commit: 167ac7294a2dc400821e4336f0cfc4de621efe97
For a collect rendezvous, call the collect-notify handler in
the main thread if it is active. A collect-notify handler can
then make sure the main thread is active and try again, if
that's useful to an application.
original commit: 0bc286e81827f029dd02a3627a192edd053b3b91
- the collector now releases bignum temporaries in the collector
rather than relocating them so we don't keep around huge bignum
temporaries forever.
gc.c
- removed the presumably useless vector-handling code from load()
which used to be required to handle fasl groups.
scheme.c
- object files are no longer compressed as a whole, and the parameter
compile-compressed is no longer defined. instead, the individual
fasl objects within an object file are compressed whenever the
new parameter fasl-compressed is set to its default value, #t.
this allows the fasl reader to seek past portions of an object
file that are not of interest, i.e., visit-only code and data
when "revisiting" an object file and revisit-only code and data
when "visiting" an object file. the compressed portions are
compressed using the format and level specified by the compress-format
and compress-level parameters. the C-coded fasl reader and
boot-file loader no longer handle compressed files; these are
handled, less efficiently, by the Scheme entry point (fasl-read).
a warning exception is raised the first time a program attempts
to create or read a compressed fasl file.
7.ss, s/Mf-base, back.ss, bytevector.ss, cmacros.ss, compile.ss,
fasl-helpers.ss, fasl.ss, primdata.ss, strip.ss, syntax.ss,
externs.h, fasl.c, gc.c, scheme.c, thread.c,
mats/6.ms, mats/7.ms, mats/bytevector.ms, mats/misc.ms, patch*,
root-experr*,
intro.stex, use.stex, io.stex, system.stex,
release_notes.stex
- added begin wrappers around many of the Scheme source files that
contained multiple expressions to cut down the number of top-level
fasl objects and increase compressibility. also removed the
string filenames for debugging at the start of each file that had
one---these are best inserted universally by a modified compile-file
during a debugging session when desired. also removed unnecessary
top-level placeholder definitions for the assignments that follow.
4.ss, 5_1.ss, 5_2.ss, 5_3.ss, 5_7.ss, 6.ss, 7.ss, bytevector.ss,
cafe.ss, cback.ss, compile.ss, cp0.ss, cpcommonize.ss, cpletrec.ss,
cpnanopass.ss, cprep.ss, cpvalid.ss, date.ss, engine.ss, enum.ss,
env.ss, event.ss, exceptions.ss, expeditor.ss, fasl.ss, foreign.ss,
format.ss, front.ss, ftype.ss, inspect.ss, interpret.ss, io.ss,
library.ss, mathprims.ss, newhash.ss, pdhtml.ss, pretty.ss,
prims.ss, primvars.ss, print.ss, read.ss, record.ss, reloc.ss,
strnum.ss, syntax.ss, trace.ss
original commit: b7f161bf2939dfedce8accbfa82b92dbe011d32a
- added invoke-library
syntax.ss, primdata.ss,
8.ms, root-experr*,
libraries.stex, release_notes.stex
- updated the date
release_notes.stex
- libraries contained within a whole program or library are now
marked pending before their invoke code is run so that invoke
cycles are reported as such rather than as attempts to invoke
while still loading.
compile.ss, syntax.ss, primdata.ss,
7.ms, root-experr*
- the library manager now protects against unbound references
from separately compiled libraries or programs to identifiers
ostensibly but not actually exported by (invisible) libraries
that exist only locally within a whole program. this is done by
marking the invisibility of the library in the library-info and
propagating it to libdesc records; the latter is checked upon
library import, visit, and invoke as well as by verify-loadability.
the import and visit code of each invisible no longer complains
about invisibility since it shouldn't be reachable.
syntax.ss, compile.ss, expand-lang.ss,
7.ms, 8.ms, root-experr*, patch*
- documented that compile-whole-xxx's linearization of the
library initialization code based on static dependencies might
not work for dynamic dependencies.
system.stex
- optimized bignum right shifts so the code (1) doesn't look at
shifted-off bigits if the bignum is positive, since it doesn't
need to know in that case if any bits are set; (2) doesn't look
at shifted-off bigits if the bignum is negative if it determines
that at least one bit is set in the bits shifted off the low-order
partially retained bigit; (3) quits looking, if it must look, for
one bits as soon as it finds one; (4) looks from both ends under
the assumption that set bits, if any, are most likely to be found
toward the high or low end of the bignum rather than just in the
middle; and (5) doesn't copy the retained bigits and then shift;
rather shifts as it copies. This leads to dramatic improvements
when the shift count is large and often significant improvements
otherwise.
number.c,
5_3.ms,
release_notes.stex
- threaded tc argument through to all calls to S_bignum and
S_trunc_rem so they don't have to call get_thread_context()
when it might already have been called.
alloc.c, number.c, fasl.c, print.c, prim5.c, externs.h
- added an expand-primitive handler to partially inline integer?.
cpnanopass.ss
- added some special cases for basic arithmetic operations (+, -, *,
/, quotient, remainder, and the div/div0/mod/mod0 operations) to
avoid doing unnecessary work for large bignums when the result
will be zero (e.g,. multiplying by 0), the same as one of the
inputs (e.g., adding 0 or multiplying by 1), or the additive
inverse of one of the inputs (e.g., subtracting from 0, dividing
by -1). This can have a major beneficial affect when operating
on large bignums in the cases handled. also converted some uses
of / into integer/ where going through the former would just add
overhead without the possibility of optimization.
5_3.ss,
number.c, externs.h, prim5.c,
5_3.ms, root-experr, patch*,
release_notes.stex
- added a queue to hold pending signals for which handlers have
been registered via register-signal-handler so up to 63 (configurable
in the source code) unhandled signals are buffered before the
handler has to start dropping them.
cmacros.ss, library.ss, prims.ss, primdata.ss,
schsig.c, externs.h, prim5.c, thread.c, gc.c,
unix.ms,
system.stex, release_notes.stex
- bytevector-compress now selects the level of compression based
on the compress-level parameter. Prior to this it always used a
default setting for compression. the compress-level parameter
can now take on the new minimum in addition to low, medium, high,
and maximum. minimum is presently treated the same as low
except in the case of lz4 bytevector compression, where it
results in the use of LZ4_compress_default rather than the
slower but more effective LZ4_compress_HC.
cmacros,ss, back.ss,
compress_io.c, new_io.c, externs.h,
bytevector.ms, mats/Mf-base, root-experr*
io.stex, objects.stex, release_notes.stex
original commit: 72d90e4c67849908da900d0b6249a1dedb5f8c7f
Add a shortcut check when refiying the continuation frame in tail
position, which is significantly cheaper when the frame is already
there. We pay down the check by skipping an attachment-lists check
that is not needed if the frame is newly reified.
Aslo, add a one-shot continuation-frame cache, which makes a shallow
temporary attachment cheaper, as in
(let loop ([i N])
(if (zero? i)
0
(loop (call-setting-continuation-attachment
i
(lambda ()
(f (sub1 i)))))))
The cache is just one frame. Keeping a chain of allocated-by-not-GCed
frames doesn't pay off.
Meanwhile, remove the leftover `$shift-attachment` library entry.
original commit: 1f454f536b1d7efe20fe9e793cda31e54e31e5f4
Use OSSP UUID on {Free,Open}BSD and native UUID on NetBSD.
Building on OpenBSD requires a filesystem mounted with wxneeded.
original commit: e964d7d01a6d115e469c01626896b683d421d599
- added compress-level parameter to select a compression level for
file writing and changed the default for lz4 compression to do a
better job compressing. finished splitting glz input routines
apart from glz output routines and did a bit of other restructuring.
removed gzxfile struct-as-bytevector wrapper and moved its fd
into glzFile. moved DEACTIVATE to before glzdopen_input calls
in S_new_open_input_fd and S_compress_input_fd, since glzdopen_input
reads from the file and could block. the compress format and now
level are now recorded directly the thread context. replaced
as-gz? flag bit in compressed bytevector header word with a small
number of bits recording the compression format at the bottom of
the header word. flushed a couple of bytevector compression mats
that depended on the old representation. (these last few changes
should make adding new compression formats easier.) added
s-directory build options to choose whether to compress and, if
so, the format and level.
compress-io.h, compress-io.c, new-io.c, equates.h, system.h,
scheme.c, gc.c,
io.ss, cmacros.ss, back.ss, bytevector.ss, primdata.ss, s/Mf-base,
io.ms, mat.ss, bytevector.ms, root-experr*,
release_notes.stex, io.stex, system.stex, objects.stex
- improved the effectiveness of LZ4 boot-file compression to within
15% of gzip by increasing the lz4 output-port in_buffer size to
1<<18. With the previous size (1<<14) LZ4-compressed boot files
were about 50% larger. set the lz4 input-port in_buffer and
out_buffer sizes to 1<<12 and 1<<14. there's no clear win at
present for larger input-port buffer sizes.
compress-io.c
- To reduce the memory hit for the increased output-port in_buffer
size and the corresponding increase in computed out_buffer size,
one output-side out_buffer is now allocated (lazily) per thread
and stored in the thread context. The other buffers are now
directly a part of the lz4File_out and lz4File_in structures
rather than allocated separately.
compress-io.c, scheme.c, gc.c,
cmacros.ss
- split out the buffer emit code from glzwrite_lz4 into a
separate glzemit_lz4 helper that is now also used by gzclose
so we can avoid dealing with a NULL buffer in glzwrite_lz4.
glzwrite_lz4 also uses it to writing large buffers directly and
avoid the memcpy.
compress-io.c
- replaced lz4File_out and lz4File_in mode enumeration with the
compress format and inputp boolean. using switch to check and
raising exceptions for unexpected values to further simplify
adding new compression formats in the future.
compress-io.c
- replaced the never-defined struct lz4File pointer in glzFile
union with the more specific struct lz4File_in_r and Lz4File_out_r
pointers.
compress-io.h, compress-io.c
- added free of lz4 structures to gzclose. also changed file-close
logic generally so that (1) port is marked closed before anything is
freed to avoid dangling pointers in the case of an interrupt or
error, and (2) structures are freed even in the case of a write
or close error, before the error is reported. also now mallocing
glz and lz4 structures after possibility of errors have passed where
possible and freeing them when not.
compress-io.c,
io.ss
- added return-value checks to malloc calls and to a couple of other
C-library calls.
compress-io.c
- corrected EINTR checks to look at errno rather than return codes.
compress-io.c
- added S_ prefixes to the glz* exports
externs.h, compress-io.c, new-io.c, scheme.c, fasl.c
- added entries for mutex-name and mutex-thread
threads.stex
original commit: 722ffabef4c938bc92c0fe07f789a9ba350dc6c6
convention that the only undocumented externs are prefixed with
S_.
externs.h, stats.c, thread.c
original commit: 898c3941d807a18c393a4575a72283e3671ee7e5
allocating the thread context fails from Sactivate_thread. before
this change, the error was raised on the main thread, which resulted
in strange behavior at best. also added who argument to
S_create_thread_object to allow it to report either Sactivate_thread
or fork-thread led to the error.
externs.h, schsig.c, scheme.c, thread.c
original commit: 89691cee27ee7d5e9bffae530b6346e96f8cc7ad
