This commit does four things:
* Adds "pb.ss" and "pb.c", which implement a portable bytecode
backend and interpreter that is intended for bootstrapping. A
single set of pb bootfiles can support bootstrapping on all
platforms --- as long as the C compiler supports a 64-bit integer
type. The pb machine supports foreign calls for only a small set of
recognized prototypes, and it does not support foriegn callables.
Use `./configure --pb` to build the pb variant.
* Changes the kernel's casts between `ptr` and `void*` types. In a pb
build, the `ptr` type can be a 64-bit integer type while `void*` is
a 32-bit pointer type, so casts must go through an intermediate
integer type.
* Adjusts the compiler to accomodate run-time-determined endianness.
Making the compiler agnostic to word size is not practical, but
only a few pieces depend on the target machine's endianness, and
those can generally be deferred to a run-time choice of byte-based
operations. The one exception is that ftype bit fields are not
allowed unless accompanied by an explicit endianness declaration.
* Start reducing duplication among platform-specific makefiles. For
example, `Mf-ta6osx` chains to `Mf-a6osx` to avoid repeating most
of it. A lot more can be done here.
original commit: 97533fa9d8b8400b0dc1a890768c7d30c91257e0
"Externals" supports fasling with some values lifted out an provided
separately.
Lifting the restriction on source file descriptor paths, formerly to
strings, means that paths can be represented in a different way, and
they can be fasled through a different means than the built-in
encodings.
original commit: b6b0ae67b08f2e9bc8b7fafe5ebad0375b6ce9db
Merge changes in the way that fasl streams are compressed. The new
approach makes compression explicit in the fasl representation, which
means that tricks like uzing zcat on a fasl file will no longer work
(at least not efficiently).
original commit: 167ac7294a2dc400821e4336f0cfc4de621efe97
Besides adding supportt for `__collect-safe` and other repairs,
introduce a write-write fence with the write barrier, which is
intended to avoid one thread using an object created in another thread
before the object's initializing writes are visible.
original commit: 543bd16739c08e5a8f88c470b52db0f23a27d260
Besides updating for unboxed floating point, the ppc32 build uses a
return register, and the continuation-attachments implementation was
not right for that mode.
original commit: dd2d01fb26ace819c73f258b9b53739f9dda1d34
Flonum operations like `fltruncate` and `flsin` are implemented by
calling functions from the C library. Unboxing these involves a
generalazation the `foreign-call` intermediate form to handle unboxing
and to work in a non-tail position (especially by telling the register
allocator that caller-saved registers will be trashed). An internal
'atomic convention on a foreign call indicates that no callback into
Scheme is possible, so some setup/teardown (including stashing
callee-saved registers) can be skipped.
original commit: fd89919634d0d5272e046b47bb81bcc66e22a741
This is a follow-up to 276f8da076, where `(%tc-ref cp)` was supposed
to be preserved by moving it into %cp, but intrinisics for bytevector
arguments can kill %cp. Use a temporary to expose things properly to
the register allocator.
original commit: 3a29db06a452e46e69ebcde524b3b9acb435dec3
* Fix calculation of segment index for 32-bit platforms
* Fix allocation of mark-bit and list-bit arrays in certain unusual
cases.
* Fix dirty sweep of records on marked pages that have non-pointer
fields.
* Fix allocation of eveen-sized immobile vectors; a pad word needs to
be cleared.
* Fix and extend the heap checker (which was used to find several of
the other problems).
original commit: 8b5e65f5eafac5aea7394901e1dd2f2fc3ccf2bd
Part of the repair makes it ok to re-sweep an ephemeron, which is more
consistent with evertything else.
original commit: 2c11bb39129b1492108390a704eb08deaa5d6bcc
The C compiler doesn't generate a tail call in a place where I
expected one, and maybe it's better to branch at the call site anyway.
original commit: 70fa8e7f7bd891c548c877cabdd15073aa2aa01b
Change the GC so that it can mark and sweep objects in-place, instead
of always copying. This change is helpful for reducing peak memory
use while performing a collection on a large, old heap.
Some non-copying support was already in place for locked objects,
but the new implementation is faster and more general. As an
alternative to locking, the storage manager now provides "immobile"
allocation (currently only for bytevectors, vectors, and boxes),
which allocates an object that won't move but that can be GCed if
it's not referenced. A locked object is an object that has been
immobiled and that is on a global list --- mostly the old,
non-scalable implementation of locked objects brought back, since
immobile objects cover the cases that need to scale.
original commit: aecb7b736cb1d52764c292fa6364a674958dfde3
When collecting to the maximum generation with object counts enabled,
a structure type would effectively become permanently reachable.
Also, add `bytes-finalized` to report how many bytes were associated
with guardian-based finalization by the most recent collection.
original commit: 852f5e2de95a26d3500321c4d4d732407945a57a
Replace repetitive C code in "gc.c" and "vfasl.c" with an
implementation using a little "Parenthe-C" language, which is a
somewhat declarative description of object tracing. From that
descrition, we generate different kinds of tracing functions, such as
the copy function or the sweep function.
The little language is still bascially C, just with parentheses and
parameterization that is much better than trying to use the C
preprocessor. (The "mkgc.ss" file includes the compiler from
Parenthe-C to C.)
Besides replacing existing code, we also generate a new traversal to
implement `compute-object-sizes`. Finally, the GC can now perform a
fused `collect` and `compute-object-sizes` in a single traversal.
Also improve the way that locked objects are detected during GC. This
can make a significant difference (on the order of 10-20% for a full
collection) when locked objects are long-lived.
original commit: de1f5c41d729ac75822a1f1e633ec6d042c883dc
Only half(!) of the needed space was actually allocated. The extra
space is ony used after a GC, however, and a GC makes the extra room,
so that's why things haven't fallen over completely, but that's more
subtle than intended.
original commit: 3d72bc14b9247d6764809cb651403dbb4063a905
For a collect rendezvous, call the collect-notify handler in
the main thread if it is active. A collect-notify handler can
then make sure the main thread is active and try again, if
that's useful to an application.
original commit: 0bc286e81827f029dd02a3627a192edd053b3b91
This operation effectively allows sending an expression back to a
continuation, instead of just a value. It's the same as Marc Feeley's
`continuation-slice` operation, but adjusted slightly to support
continuation attachments.
original commit: d0e36e72d20a6eaa5d9d8b795da5e77abde75289
While "\44\26\2\f6" currently works as a terminator for non-compressed
fasl streams, the working byte sequence varies as the fasl format
changes. Add "\177" as a simpler and unchanging terminator.
original commit: 332019360491be6cedd2063c9a8056183d764bbb
- the collector now releases bignum temporaries in the collector
rather than relocating them so we don't keep around huge bignum
temporaries forever.
gc.c
- removed the presumably useless vector-handling code from load()
which used to be required to handle fasl groups.
scheme.c
- object files are no longer compressed as a whole, and the parameter
compile-compressed is no longer defined. instead, the individual
fasl objects within an object file are compressed whenever the
new parameter fasl-compressed is set to its default value, #t.
this allows the fasl reader to seek past portions of an object
file that are not of interest, i.e., visit-only code and data
when "revisiting" an object file and revisit-only code and data
when "visiting" an object file. the compressed portions are
compressed using the format and level specified by the compress-format
and compress-level parameters. the C-coded fasl reader and
boot-file loader no longer handle compressed files; these are
handled, less efficiently, by the Scheme entry point (fasl-read).
a warning exception is raised the first time a program attempts
to create or read a compressed fasl file.
7.ss, s/Mf-base, back.ss, bytevector.ss, cmacros.ss, compile.ss,
fasl-helpers.ss, fasl.ss, primdata.ss, strip.ss, syntax.ss,
externs.h, fasl.c, gc.c, scheme.c, thread.c,
mats/6.ms, mats/7.ms, mats/bytevector.ms, mats/misc.ms, patch*,
root-experr*,
intro.stex, use.stex, io.stex, system.stex,
release_notes.stex
- added begin wrappers around many of the Scheme source files that
contained multiple expressions to cut down the number of top-level
fasl objects and increase compressibility. also removed the
string filenames for debugging at the start of each file that had
one---these are best inserted universally by a modified compile-file
during a debugging session when desired. also removed unnecessary
top-level placeholder definitions for the assignments that follow.
4.ss, 5_1.ss, 5_2.ss, 5_3.ss, 5_7.ss, 6.ss, 7.ss, bytevector.ss,
cafe.ss, cback.ss, compile.ss, cp0.ss, cpcommonize.ss, cpletrec.ss,
cpnanopass.ss, cprep.ss, cpvalid.ss, date.ss, engine.ss, enum.ss,
env.ss, event.ss, exceptions.ss, expeditor.ss, fasl.ss, foreign.ss,
format.ss, front.ss, ftype.ss, inspect.ss, interpret.ss, io.ss,
library.ss, mathprims.ss, newhash.ss, pdhtml.ss, pretty.ss,
prims.ss, primvars.ss, print.ss, read.ss, record.ss, reloc.ss,
strnum.ss, syntax.ss, trace.ss
original commit: b7f161bf2939dfedce8accbfa82b92dbe011d32a
Commit 72d90e4 ("library-manager, numeric, and bytevector-compres
improvements") introduced a regression causing Win32 nmake builds to
fail due to an undefined symbol in c\schsig.c.
This symbol (scheme_signals_registered) is defined in a preprocessor-
conditional code block excluding WIN32 between lines 544 and 737.
This commit bypasses the build regression by enclosing the unresolved
expression in a preprocessor conditional excluding WIN32.
See GitHub Issue #497 for more details:
https://github.com/cisco/ChezScheme/issues/497
original commit: 10bccf39badee76f80d87326d2fc7c4d808fa08e
- added invoke-library
syntax.ss, primdata.ss,
8.ms, root-experr*,
libraries.stex, release_notes.stex
- updated the date
release_notes.stex
- libraries contained within a whole program or library are now
marked pending before their invoke code is run so that invoke
cycles are reported as such rather than as attempts to invoke
while still loading.
compile.ss, syntax.ss, primdata.ss,
7.ms, root-experr*
- the library manager now protects against unbound references
from separately compiled libraries or programs to identifiers
ostensibly but not actually exported by (invisible) libraries
that exist only locally within a whole program. this is done by
marking the invisibility of the library in the library-info and
propagating it to libdesc records; the latter is checked upon
library import, visit, and invoke as well as by verify-loadability.
the import and visit code of each invisible no longer complains
about invisibility since it shouldn't be reachable.
syntax.ss, compile.ss, expand-lang.ss,
7.ms, 8.ms, root-experr*, patch*
- documented that compile-whole-xxx's linearization of the
library initialization code based on static dependencies might
not work for dynamic dependencies.
system.stex
- optimized bignum right shifts so the code (1) doesn't look at
shifted-off bigits if the bignum is positive, since it doesn't
need to know in that case if any bits are set; (2) doesn't look
at shifted-off bigits if the bignum is negative if it determines
that at least one bit is set in the bits shifted off the low-order
partially retained bigit; (3) quits looking, if it must look, for
one bits as soon as it finds one; (4) looks from both ends under
the assumption that set bits, if any, are most likely to be found
toward the high or low end of the bignum rather than just in the
middle; and (5) doesn't copy the retained bigits and then shift;
rather shifts as it copies. This leads to dramatic improvements
when the shift count is large and often significant improvements
otherwise.
number.c,
5_3.ms,
release_notes.stex
- threaded tc argument through to all calls to S_bignum and
S_trunc_rem so they don't have to call get_thread_context()
when it might already have been called.
alloc.c, number.c, fasl.c, print.c, prim5.c, externs.h
- added an expand-primitive handler to partially inline integer?.
cpnanopass.ss
- added some special cases for basic arithmetic operations (+, -, *,
/, quotient, remainder, and the div/div0/mod/mod0 operations) to
avoid doing unnecessary work for large bignums when the result
will be zero (e.g,. multiplying by 0), the same as one of the
inputs (e.g., adding 0 or multiplying by 1), or the additive
inverse of one of the inputs (e.g., subtracting from 0, dividing
by -1). This can have a major beneficial affect when operating
on large bignums in the cases handled. also converted some uses
of / into integer/ where going through the former would just add
overhead without the possibility of optimization.
5_3.ss,
number.c, externs.h, prim5.c,
5_3.ms, root-experr, patch*,
release_notes.stex
- added a queue to hold pending signals for which handlers have
been registered via register-signal-handler so up to 63 (configurable
in the source code) unhandled signals are buffered before the
handler has to start dropping them.
cmacros.ss, library.ss, prims.ss, primdata.ss,
schsig.c, externs.h, prim5.c, thread.c, gc.c,
unix.ms,
system.stex, release_notes.stex
- bytevector-compress now selects the level of compression based
on the compress-level parameter. Prior to this it always used a
default setting for compression. the compress-level parameter
can now take on the new minimum in addition to low, medium, high,
and maximum. minimum is presently treated the same as low
except in the case of lz4 bytevector compression, where it
results in the use of LZ4_compress_default rather than the
slower but more effective LZ4_compress_HC.
cmacros,ss, back.ss,
compress_io.c, new_io.c, externs.h,
bytevector.ms, mats/Mf-base, root-experr*
io.stex, objects.stex, release_notes.stex
original commit: 72d90e4c67849908da900d0b6249a1dedb5f8c7f
A 0 relocation is used by fcallable code as a recognizable cookie, and
its relocations must be preserved.
original commit: 38fb3fdf75cf6540d6bd2568f015af6272d22995
Instead of constaining the use of event-detour so much, make it merely
unlikely that the detour will have to allocate when used in a loop
that otherwise doesn't allocate. We'll only have to allocate if the
available stack space turns out to be too small --- and if we do
allocate, it's not the end of the world.
original commit: f1dbed82df415c18c8304bedcee2ecf4912badc7
Having the trap check allocate is questionable, since it can be
triggered during a loop that otherwise performs no allocation. Also,
on platforms where at most 1 argument is passed in a register, then
sending two arguments to the event handler could potentially need
stack space that isn't there. So, constrain the smaller trap-check
code to cases where no stack space is needed and where no allocation
happens unless the wrong number of arguments are provided.
original commit: 260a7ef5bc0bf851d9848587b0a78bdb4aab59f8
When a proceudre starts with a trap check, move the check to the very
beginning, even before checking the argument count. That way, event
detection can turn into a compact jump to an event handler, instead of
inserting a general call to `$event` in the procedure body.
original commit: 06b12d505698a2378734689370bb9e0f8eda06b9
Fix 'reloc to avoid a crash on static-generation code, and add
'reloc+offset to report an offset for each entry.
original commit: 4d4195044377f9c619cfb46056e365044069d5bc
In the general form of a function call, the return point embeds 4
words of information: offset to the start of the enclosing function,
frame size, live-veriable mask, and multiple-value return address. In
the common case, however, the multiple-value return address is either
the same as the return address or it is a `values-error` library
function, and the frame size and live-variable mask fit into a word
with bits to spare. This patch implements a more compact return point
for that common case, which shrinks the 4 words to 2 and also avoids a
relocation (= 1 more word).
Multiple-value returns are more complex with this change (i.e.,
require more code), since they must check whether the return point is
compact or not. But multiple-value returns are far less common than
function calls, so saving function-call space is a clear win.
Overall, this change tends to reduce code size by about 10% on x86_64.
original commit: 1f53b5eabef966db01086cb32e544bbf8deacfca
