- added invoke-library
syntax.ss, primdata.ss,
8.ms, root-experr*,
libraries.stex, release_notes.stex
- updated the date
release_notes.stex
- libraries contained within a whole program or library are now
marked pending before their invoke code is run so that invoke
cycles are reported as such rather than as attempts to invoke
while still loading.
compile.ss, syntax.ss, primdata.ss,
7.ms, root-experr*
- the library manager now protects against unbound references
from separately compiled libraries or programs to identifiers
ostensibly but not actually exported by (invisible) libraries
that exist only locally within a whole program. this is done by
marking the invisibility of the library in the library-info and
propagating it to libdesc records; the latter is checked upon
library import, visit, and invoke as well as by verify-loadability.
the import and visit code of each invisible no longer complains
about invisibility since it shouldn't be reachable.
syntax.ss, compile.ss, expand-lang.ss,
7.ms, 8.ms, root-experr*, patch*
- documented that compile-whole-xxx's linearization of the
library initialization code based on static dependencies might
not work for dynamic dependencies.
system.stex
- optimized bignum right shifts so the code (1) doesn't look at
shifted-off bigits if the bignum is positive, since it doesn't
need to know in that case if any bits are set; (2) doesn't look
at shifted-off bigits if the bignum is negative if it determines
that at least one bit is set in the bits shifted off the low-order
partially retained bigit; (3) quits looking, if it must look, for
one bits as soon as it finds one; (4) looks from both ends under
the assumption that set bits, if any, are most likely to be found
toward the high or low end of the bignum rather than just in the
middle; and (5) doesn't copy the retained bigits and then shift;
rather shifts as it copies. This leads to dramatic improvements
when the shift count is large and often significant improvements
otherwise.
number.c,
5_3.ms,
release_notes.stex
- threaded tc argument through to all calls to S_bignum and
S_trunc_rem so they don't have to call get_thread_context()
when it might already have been called.
alloc.c, number.c, fasl.c, print.c, prim5.c, externs.h
- added an expand-primitive handler to partially inline integer?.
cpnanopass.ss
- added some special cases for basic arithmetic operations (+, -, *,
/, quotient, remainder, and the div/div0/mod/mod0 operations) to
avoid doing unnecessary work for large bignums when the result
will be zero (e.g,. multiplying by 0), the same as one of the
inputs (e.g., adding 0 or multiplying by 1), or the additive
inverse of one of the inputs (e.g., subtracting from 0, dividing
by -1). This can have a major beneficial affect when operating
on large bignums in the cases handled. also converted some uses
of / into integer/ where going through the former would just add
overhead without the possibility of optimization.
5_3.ss,
number.c, externs.h, prim5.c,
5_3.ms, root-experr, patch*,
release_notes.stex
- added a queue to hold pending signals for which handlers have
been registered via register-signal-handler so up to 63 (configurable
in the source code) unhandled signals are buffered before the
handler has to start dropping them.
cmacros.ss, library.ss, prims.ss, primdata.ss,
schsig.c, externs.h, prim5.c, thread.c, gc.c,
unix.ms,
system.stex, release_notes.stex
- bytevector-compress now selects the level of compression based
on the compress-level parameter. Prior to this it always used a
default setting for compression. the compress-level parameter
can now take on the new minimum in addition to low, medium, high,
and maximum. minimum is presently treated the same as low
except in the case of lz4 bytevector compression, where it
results in the use of LZ4_compress_default rather than the
slower but more effective LZ4_compress_HC.
cmacros,ss, back.ss,
compress_io.c, new_io.c, externs.h,
bytevector.ms, mats/Mf-base, root-experr*
io.stex, objects.stex, release_notes.stex
original commit: 72d90e4c67849908da900d0b6249a1dedb5f8c7f
In previous versions of Chez Scheme, multiple object files could be
combined by concatinating them into a single file. To support faster
object file loading and loadability verification, recompile information
and information about libraries and top-level programs within an object
file is now placed at the top of the file. The new
concatenate-object-files procedure can be used to combine multiple object
files while moving this information to the top of the combined file.
original commit: d4ef2ad9393578ff3ffe3b712736bc6a4ae7b8eb
loadability without actually loading; also, support for unregistering
guarded objects.
- improved error reporting for library compilation-instance errors:
now including the name of the object file from which the "wrong"
compilation instance was loaded, if it was loaded from (or compiled
to) an object file and the original importing library, if it was
previously loaded from an object file due to a library import.
syntax.ss, 7.ss, interpret.ss,
8.ms, root-experr*
- removed situation and for-input? arguments from $make-load-binary,
since the only consumer always passes 'load and #f.
7.ss,
scheme.c
- $separate-eval now prints the stderr and stdout of the subprocess
to help in diagnosing separate-eval and separate-compile issues.
mat.ss
- added unregister-guardian, which can be used to unregister
the unressurected objects registered with any guardian. guardian?
can be used to distinguish guardian procedures from other objects.
cp0.ss, cmacros.ss, cpnanopass.ss, ftype.ss, primdata.ss,
prims.ss,
gcwrapper.c, prim.c, externs.h,
4.ms, primvars.ms
release_notes.stex
smgmt.stex, threads.stex
- added verify-loadability. given a situation (visit, revisit,
or load) and zero or more pathnames (each of which may be optionally
paired with a library search path), verity-loadability checks
whether the set of object files named by those pathnames and any
additional object files required by library requirements in the
given situation can be loaded together. it raises an exception
in each case where actually attempting to load the files would
raise an exception and additionally in cases where loading files
would result in the compilation or loading of source files in
place of the object files. if the check is successful,
verity-loadability returns an unspecified value. in either case,
although portions of the object files are read, none of the
information read from the object files is retained, and none of
the object code is read, so there are no side effects other than
the file operations and possibly the raising of an exception.
library and program info records are now moved to the top of each
object file produced by one of the file compilation routines,
just after recompile info, with a marker to allow verity-loadability
to stop reading once it reads all such records. this change is
not entirely backward compatible; the repositioning of the records
can be detected by a call to list-library made from a loaded file
before the definition of one or more libraries. it is fully
backward compatible for typical library files that contain a
single library definition and nothing else. adding this feature
required changes to the object-file format and corresponding
changes in the compiler and library manager. it also required
moving cross-library optimization information from library/ct-info
records (which verity-loadability must read) to the invoke-code
for each library (which verity-loadability does not read) to
avoid reading and permanently associating record-type descriptors
in the code with their uids.
compile.ss, syntax.ss, expand-lang.ss, primdata.ss, 7.ss,
7.ms, misc.ms, root-experr*, patch*,
system.stex, release_notes.stex
- fixed a bug that bit only with the compiler compiled at
optimize-level 2: add-library/rt-records was building a library/ct-info
wrapper rather than a library/rt-info wrapper.
compile.ss
- fixed a bug in visit-library that could result in an indefinite
recursion: it was not checking to make sure the call to $visit
actually added compile-time info to the libdesc record. it's not
clear, however, whether the libdesc record can be missing
compile-time information on entry to visit-library, so the code
that calls $visit (and now checks for compile-time information
having been added) might not be reachable. ditto for
revisit-library.
syntax.ss
syntax.ss, primdata.ss,
7.ms, root-experr*, patch*,
system.stex, release_notes.stex
- added some argument-error checks for library-directories and
library-extensions, and fixed up the error messages a bit.
syntax.ss,
7.ms, root-experr*
- compile-whole-program now inserts the program record into the
object file for the benefit of verify-loadability.
syntax.ss,
7.ms, root-experr*
- changed 'loading' import-notify messages to the more precise
'visiting' or 'revisiting' in a couple of places.
syntax.ss,
7.ms, 8.ms
original commit: b911ed47190727b0e1d6a88c0e473d1757accdcd
Allow a library-defined function to be inlined when the inlined
expressions refer to other library-defined functions. Since the
library function's body may already have inlined calls, don't allow
further inlining of calls within the inlined code.
This commit also adds `$app/no-inline`, which can be used to prevent
inlining of a function. For consumers other than Racket on Chez
Scheme, probably it would make sense to provide a nicer-looking
syntactic form that expands to use the internal `$app/no-inline`
function.
original commit: 628d57e1bd2e658aad4da97a3e85bda72c38f6ab
On x86_64, a POPCNT instruction is usually available, and it can speed
up `fxpopcount` operations by a factor of 2-3.
Since POPCNT isn't always available, code using `fxpopcount` is
compiled to a call to a generic implementation. The linker substitutes
a POPCNT instruction when it determines at runtime that POPCNT is
available.
Some measurements on a 2018 MacBook Pro (2.7 GHz Core i7) using the
program below:
popcnt = this implementation, POPCNT discovered
nocnt = this implementation, POPCNT considered unavailable
optcnt = compile to use POPCNT directly (no linker work)
cpcnt = compile to inlined generic (no linker work, no POPCNT)
Since the generic implementation is always a 64-bit popcount, it's not
as good as an inlined version for `fxpopcount32`, but otherwise the
link-edit approach to POPCNT works well:
fxpopcount fxpopcount32
popcnt: 0.098s
nocnt: 0.284s
optcnt 0.109s [slower means noise?]
cpcnt: 0.279s 0.188s
(optimize-level 3)
(time
(let loop ([v #f] [i 100000000])
(if (fx= i 0)
v
(loop (fxpopcount i) (fx- i 1)))))
original commit: 5f090e509f8fe5edc777ed9f0463b20c2e571336
Don't run cptypes, when cp0 is disabled, for example with
(run-cp0 (lamba (cp0 x) x)
This is easier to understand because run-cp0 is a single point to control
all the cp reductions. The reductions in cptypes can be independently disable
using enable-type-recovery.
original commit: b23645e669fbf02806a261a2d87160fdbe06db93
and functionality improvements (including support for measuring
coverage), primitive argument-checking fixes, and object-file changes
resulting in reduced load times (and some backward incompatibility):
- annotations are now preserved in object files for debug
only, for profiling only, for both, or not at all, depending
on the settings of generate-inspector-information and
compile-profile. in particular, when inspector information
is not enabled but profiling is, source information does
not leak into error messages and inspector output, though it is
still available via the profile tools. The mechanics of this
involved repurposing the fasl a? parameter to hold an annotation
flags value when it is not #f and remaking annotations with
new flags if necessary before emitting them.
compile.ss, fasl.ss, misc.ms
- altered a number of mats to produce correct results even
when the 's' directory is profiled.
misc.ms, cp0.ms, record.ms
- profile-release-counters is now generation-friendly; that is,
it doesn't look for dropped code objects in generations that have
not been collected since the last call to profile-release-counters.
also, it no longer allocates memory when it releases counters.
pdhtml.ss,
gc.c, gcwrapper.c, globals.h, prim5.c
- removed unused entry points S_ifile, S_ofile, and S_iofile
alloc.c, externs.h
- mats that test loading profile info into the compiler's database
to guide optimization now weed out preexisting entries, in case
the 's' directory is profiled.
4.ms, mat.ss, misc.ms, primvars.ms
- counters for dropped code objects are now released at the start
of each mat group.
mat.ss
- replaced ehc (enable-heap-check) option with hci (heap-check-interval)
option that allows heap checks to be performed periodically rather
than on each collection. hci=0 is equivalent to ehc=f (disabling
heap checks) and hci=1 is equivalent to ehc=t (enabling heap
checks every collection), while hci=100 enables heap checks only
every 100th collection. allx and bullyx mats use this feature
to reduce heap-checking overhead to a more reasonable level. this
is particularly important when the 's' directory is profiled,
since the amount of static memory to be checked is greatly increased
due to the counters.
mats/Mf-base, mat.ss, primvars.ms
- added a mat that calls #%show-allocation, which was otherwise not
being tested.
misc.ms
- removed a broken primvars mat and updated two others. in each case,
the mat was looking for information about primitives in the wrong
(i.e., old) place and silently succeeding when it didn't find any
primitives to tests. the revised mats (along with a few others) now
check to make sure at least one identifier has the information they
look for. the removed mat was checking for library information that
is now compiled in, so the mat is now unnecessary. the others were
(not) doing argument-error checks. fixing these turned up a handful of
problems that have also been fixed: a couple of unbound variables in the
mat driver, two broken primdata declarations, a tardy argument check
by profile-load-data, and a bug in char-ready?, which was requiring
an argument rather than defaulting it to the current input port.
primdata.ss, pdhtml.ss, io.ms,
primdvars.ms, 4.ms, 6.ms, misc.ms, patch*
- added initial support for recording coverage information. when the
new parameter generate-covin-files is set, the compiler generates
.covin files containing the universe of all source objects for which
profile forms are present in the expander output. when profiling
and generation of covin files are enabled in the 's' directory, the
mats optionally generate .covout files for each mat file giving
the subset of the universe covered by the mat file, along with an
all.covout in each mat output directory aggregating the coverage
for the directory and another all.covout in the top-level mat
directory aggregating the coverage for all directories.
back.ss, compile.ss, cprep.ss, primdata.ss, s/Mf-base,
mat.ss, mats/Mf-base, mats/primvars.ms
- support for generating covout files is now built in. with-coverage-output
gathers and dumps coverage information, and aggregate-coverage-output
combines (aggregates) covout files.
pdhtml.ss, primdata.ss, compile.ss,
mat.ss, mats/Mf-base, primvars.ms
- profile-clear now adjusts active coverage trackers to avoid losing
coverage information.
pdhtml.ss,
prim5.c
- nested with-coverage calls are now supported.
pdhtml.ss
- switched to a more compact representation for covin and covout files;
reduces disk space (compressed or not) by about a factor of four
and read time by about a factor of two with no increase in write time.
primdata.ss, pdhtml.ss, cprep.ss, compile.ss,
mat.ss, mats/Mf-base
- added support for determining coverage for an entire run, including
coverage for expressions hit during boot time. 'all' mats now produce
run.covout files in each output directory, and 'allx' mats produce
an aggregate run.covout file in the mat directory.
pdhtml.ss,
mat.ss, mats/Mf-base
- profile-release-counters now adjusts active coverage trackers to
account for the counters that have been released.
pdhtml.ss,
prim5.c
- replaced the artificial "examples" target with a real "build-examples"
target so make won't think it always has to mats that depend upon
the examples directory having been compiled. mats make clean now
runs make clean in the examples directory.
mats/Mf-base
importing a library from an object file now just visits the object
file rather than doing a full load so that the run-time code for
the library is not retained. The run-time code is still read
because the current fasl format forces the entire file to be read,
but not retaining the code can lower heap size and garbage-collection
cost, particularly when many object-code libraries are imported.
The downside is that the file must be revisited if the run-time
code turns out to be required. This change exposed several
places where the code was failing to check if a revisit is needed.
syntax.ss,
7.ms, 8.ms, misc.ms, root-experr*
- fixed typos: was passing unquoted load rather than quoted load
to $load-library along one path (where it is loading source code
and therefore irrelevant), and was reporting src-path rather than
obj-path in a message about failing to define a library.
syntax.ss
- compile-file and friends now put all recompile information in
the first fasl object after the header so the library manager can
find it without loading the entire fasl file. The library manager
now does so. It also now checks to see if library object files
need to be recreated before loading them rather than loading them and
possibly recompiling them after discovering they are out of date, since
the latter requires loading the full object file even if it's out of
date, while the former takes advantage of the ability to extract just
recompile information. as well as reducing overhead, this eliminates
possibly undesirable side effects, such as creation and registration
of out-of-date nongenerative record-type descriptors. because the
library manager expects to find recompile information at the front of
an object file, it will not find all recompile information if object
files are "catted" together. also, compile-file has to hold in memory
the object code for all expressions in the file so that it can emit the
unified recompile information, rather than writing to the object file
incrementally, which can significantly increase the memory required
to compile a large file full of individual top-level forms. This does
not affect top-level programs, which were already handled as a whole,
or a typical library file that contains just a single library form.
compile.ss, syntax.ss
- the library manager now checks include files before library dependencies
when compile-imported-libraries is false (as it already did when
compile-imported-libraries is true) in case a source change affects
the set of imported libraries. (A library change can affect the set
of include files as well, but checking dependencies before include
files can cause unneeded libraries to be loaded.) The include-file
check is based on recompile-info rather than dependencies, but the
library checks are still based on dependencies.
syntax.ss
- fixed check for binding of scheme-version. (the check prevents
premature treatment of recompile-info records as Lexpand forms
to be passed to $interpret-backend.)
scheme.c
- strip-fasl-file now preserves recompile-info when compile-time info
is stripped.
strip.ss
- removed include-req* from library/ct-info and ctdesc records; it
is no longer needed now that all recompile information is maintained
separately.
expand-lang.ss, syntax.ss, compile.ss, cprep.ss, syntax.ss
- changed the fasl format and reworked a lot of code in the expander,
compiler, fasl writer, and fasl reader to allow the fasl reader
to skip past run-time information when it isn't needed and
compile-time information when it isn't needed. Skipping past
still involves reading and decoding when encrypted, but the fasl
reader no longer parses or allocates code and data in the portions
to be skipped. Side effects of associating record uids with rtds
are also avoided, as are the side effects of interning symbols
present only in the skipped data. Skipping past code objects
also reduces or eliminates the need to synchronize data and
instruction caches. Since the fasl reader no longer returns
compile-time (visit) or run-time (revisit) code and data when not
needed, the fasl reader no longer wraps these objects in a pair
with a 0 or 1 visit or revisit marker. To support this change,
the fasl writer generates separate top-level fasl entries (and
graphs) for separate forms in the same top-level source form
(e.g., begin or library). This reliably breaks eq-ness of shared
structure across these forms, which was previously broken only
when visit or revisit code was loaded at different times (this
is an incompatible change). Because of the change, fasl "groups"
are no longer needed, so they are no longer handled.
7.ss, cmacros.ss, compile.ss, expand-lang.ss, strip.ss,
externs.h, fasl.c, scheme.c,
hash.ms
- the change above is surfaced in an optional fasl-read "situation"
argument (visit, revisit, or load). The default is load. visit
causes it to skip past revisit code and data; revisit causes it
to skip past visit code and data; and load causes it not to skip
past either. visit-revisit data produced by (eval-when (visit
revisit) ---) is never skipped.
7.ss, primdata.ss,
io.stex
- to improve compile-time and run-time error checking, the
Lexpand recompile-info, library/rt-info, library-ct-info, and
program-info forms have been replaced with list-structured forms,
e.g., (recompile-info ,rcinfo).
expand-lang.ss, compile.ss, cprep.ss, interpret.ss, syntax.ss
- added visit-compiled-from-port and revisit-compiled-from-port
to complement the existing load-compiled-from-port.
7.ss, primdata.ss,
7.ms,
system.stex
- increased amount read when seeking an lz4-encrypted input
file from 32 to 1024 bytes at a time
compress-io.c
- replaced the fasl a? parameter value #t with an "all" flag value
so it's value is consistently a mask.
cmacros.ss, fasl.ss, compile.ss
- split off profile mats into a separate file
misc.ms, profile.ms (new), root-experr*, mats/Mf-base
- added coverage percent computations to mat allx/bullyx output
mat.ss, mats/Mf-base, primvars.ms
- replaced coverage tables with more generic and generally useful
source tables, which map source objects to arbitrary values.
pdhtml.ss, compile.ss, cprep.ss, primdata.ss,
mat.ss, mats/Mf-base, primvars.ms, profile.ms,
syntax.stex
- reduced profile counting overhead by using calls to fold-left
instead of calls to apply and map and by using fixnum operations
for profile counts on 64-bit machines.
pdhtml.ss
- used a critical section to fix a race condition in the calculations
of profile counts that sometimes resulted in bogus (including
negative) counts, especially when the 's' directory is profiled.
pdhtml.ss
- added discard flag to declaration for hashtable-size
primdata.ss
- redesigned the printed representation of source tables and rewrote
get-source-table! to read and store incrementally to reduce memory
overhead.
compile.ss
- added generate-covin-files to the set of parameters preserved
by compile-file, etc.
compile.ss,
system.stex
- moved covop argument before the undocumented machine and hostop
arguments to compile-port and compile-to-port. removed the
undocumented ofn argument from compile-to-port; using
(port-name ip) instead.
compile.ss, primdata.ss,
7.ms,
system.stex
- compile-port now tries to come up with a file position to supply
to make-read, which it can do if the port's positions are character
positions (presently string ports) or if the port is positioned
at zero.
compile.ss
- audited the argument-type-error fuzz mat exceptions and fixed a
host of problems this turned up (entries follow). added #f as
an invalid argument for every type for which #f is indeed invalid
to catch places where the maybe- prefix was missing on the argument
type. the mat tries hard to determine if the condition raised
(if any) as the result of an invalid argument is appropriate and
redirects the remainder to the mat-output (.mo) file prefixed
with 'Expected error', causing them to show up in the expected
error output so developers will be encouraged to audit them in
the future.
primvars.ms, mat.ss
- added an initial symbol? test on machine type names so we produce
an invalid machine type error message rather than something
confusing like "machine type #f is not supported".
compile.ss
- fixed declarations for many primitives that were specified as
accepting arguments of more general types than they actually
accept, such as number -> real for various numeric operations,
symbol -> endianness for various bytevector operations,
time -> time-utc for time-utc->date, and list -> list-of-string-pairs
for default-library-search-handler. also replaced some of the
sub-xxxx types with specific types such as sub-symbol -> endianness
in utf16->string, but only where they were causing issues with
the primvars argument-type-error fuzz mat. (this should be done
more generally.)
primdata.ss
- fixed incorrect who arguments (was map instead of fold-right,
current-date instead of time-utc->date); switched to using
define-who/set-who! generally.
4.ss, date.ss
- append! now checks all arguments before any mutation
5_2.ss
- with-source-path now properly supplies itself as who for the
string? argument check; callers like load now do their own checks.
7.ss
- added missing integer? check to $fold-bytevector-native-ref whose
lack could have resulted in a compile-time error.
cp0.ss
- fixed typo in output-port-buffer-mode error message
io.ss
- fixed who argument (was fx< rather than fx<?)
library.ss
- fixed declaration of first source-file-descriptor argument (was
sfd, now string)
primdata.ss
- added missing article 'a' in a few error messages
prims.ss
- fixed the copy-environment argument-type error message for the list
of symbols argument.
syntax.ss
- the environment procedure now catches exceptions that occur and
reraises the exception with itself as who if the condition isn't
already a who condition.
syntax.ss
- updated experr and allx patch files for changes to argument-count
fuzz mat and fixes for problems turned up by them.
root-experr*, patch*
- fixed a couple of issues setting port sizes: string and bytevector
output port put handlers don't need room to store the character
or byte, so they now set the size to the buffer length rather
than one less. binary-file-port-clear-output now sets the index
rather than size to zero; setting the size to zero is inappropriate
for some types of ports and could result in loss of buffering and
even suppression of future output. removed a couple of redundant
sets of the size that occur immediately after setting the buffer.
io.ss
- it is now possible to return from a call to with-profile-tracker
multiple times and not double-count (or worse) any counts.
pdhtml.ss, profile.ms
- read-token now requires a file position when it is handed a
source-file descriptor (since the source-file descriptor isn't
otherwise useful), and the source-file descriptor argument can
no longer be #f. the input file position plays the same role as
the input file position in get-datum/annotations. these extra
read-token arguments are now documented.
read.ss,
6.ms,
io.stex
- the source-file descriptor argument to get-datum/annotations can
no longer be #f. it was already documented that way.
read.ss
- read-token and do-read now look for the character-positions port
flag before asking if the port has port-position, since the latter
is slightly more expensive.
read.ss
- rd-error now reports the current port position if it can be determined
when fp isn't already set, i.e., when reading from a port without
character positions (presently any non string port) and fp has not
been passed in explicitly (to read-token or get-datum/annotations).
the port position might not be a character position, but it should be
better than nothing.
read.ss
- added comment noting an invariant for s_profile_release_counters.
prim5.c
- restored accidentally dropped fasl-write formdef and dropped
duplicate fasl-read formdef
io.stex
- added a 'coverage' target that tests the coverage of the Scheme-code
portions of Chez Scheme by the mats.
Makefile.in, Makefile-workarea.in
- added .PHONY declarations for all of the targets in the top-level
and workarea make files, and renamed the create-bintar, create-rpm,
and create-pkg targets bintar, rpm, and pkg.
Makefile.in, Makefile-workarea.in
- added missing --retain-static-relocation command-line argument and
updated the date
scheme.1.in
- removed a few redundant conditional variable settings
configure
- fixed declaration of condition wait (timeout -> maybe-timeout)
primdata.ss
original commit: 88501743001393fa82e89c90da9185fc0086fbcb
improved error message for compile-whole-program and
compile-whole-library when a top-level expression is discovered while
processing a wpo file.
compile.ss
original commit: 11cee34502470d720d611ffd0799353e8663a7f1
added fix for whole program/library compilation bug with help from
@owaddell who originally reported the problem in issue 386. this bug
arises from the way the parts of the combined library, and their
binary dependencies, are invoked when one of the constituent libraries
is invoked. consider, for example, a combined library that contains
(A) and (B), where (B) depends on a binary library (C). depending on
the sort order of (A) and (B), which may be unconstrained in the
partial ordering established by library dependencies, invoking (A) may
result in the invoke code for (B) being run first, without (B) ever
being explicitly invoked. this can result in bindings required from
(C) by the invoke code in (B) to be unbound. even in the case where
(A) comes before (B) in the topological sort, if they are combined
into the same cluster, (B)'s invoke code will be run as part of
invoking (A). the solution is two part: first we extend the invoke
requirements of the first library in the cluster to include the binary
libraries that precede it in the topological sort and add a dependency
on the first library in the cluster to all of the other libraries in
the cluster. this means no matter which library in the cluster is
invoked first, it will cause the first library to be invoked, in turn
ensuring the binary libraries that precede it are invoked. when there
are multiple clusters, a dependency is added from each cluster to the
first library in the cluster that precedes it. this ensures that
invoking a library in a later cluster first, will still cause all of
the dependencies of the previous clusters to be invoked. ultimately,
these extra dependencies enforce an ordering on the invocation of the
source and binary libraries that matches the topological sort, even if
the topological sort was under constrained. to maintain the property
that import requirements are a superset of the invoke and visit
requirements, we also extend the import requirements to include the
extended invoke requirements. the import requirements are also added
to the dependency graph to further constrain the topological sort and
ensure that we do not introduce artificial cycles in the import graph.
compile.ss,
7.ms,
root-experr*, patch*
original commit: 09bba001a33a5ee9268f1e5cf0cc118e8a2eec7f
on p to account for the procedure? check at optimize-level 2.
cpletrec.ss
- fixed bug in check-prelex-flags: was hardwiring $cpcheck-prelex-flags
"after" argument to 'uncprep rather than passing along its argument.
compile.ss
- commented out local definition of sorry! so that problems detected
by $cpcheck-prelex-flags actually result in a raised exception.
cprep.ss
original commit: 674e2180d6893000446590038dcacf0ef661e564
controversial, unless I damaged something in the process of integrating
them with other recent changes. the user's guide and release notes
have been updated as well to reflect the changes of interest to end
users.
- the body of load-library is now wrapped in a $pass-time with
to show the time spent loading libraries separately from the time
spent in expand.
syntax.ss
- interpret now plays the pass-time game
interpret.ss
- added compile-time-value? predicate and
compile-time-value-value accessor
syntax.ss, primdata.ss,
8.ms, primvars.ms, root-experr*
- $pass-stats now returns accurrate stats for the currently timed
pass.
7.ss
- compile-whole-program and compile-whole-library now propagate
recompile info from the named wpo file to the object file
to support maybe-compile-program and maybe-compile-library in
the case where compile-whole-{program,library} overwrites the
original object file.
compile.ss,
7.ms, mat.ss, primvars.ms
- replaced the ancient and unusable bintar with one that creates
a useful tarball for binary installs
bintar
- generated Mf-install InstallBin (InstallLib, InstallMan) now
correctly indirects through InstallPrefix if the --installbin
(--installlib, --installman) configure flag is not present.
src/configure
- removed definition of generate-procedure-source-information
patch.ss
- guardian tconc cells are now allocated in generation 0 in the hope
that they can be released more quickly.
gc.c
- added ftype-guardian syntax: (ftype-guardian A) creates a new
guardian for ftype pointers of type A, the first base field (or
one of the first base fields in the case of unions) of which must
be a word-sized integer with native endianness representing a
reference count. ftype pointers are registered with and retrieved
from the guardian just like objects are registered with and
retrieved from any guardian. the difference is that the garbage
collector decrements the reference count before resurrecting an
ftype pointer and resurrects only those whose reference counts
become zero, i.e., are ready for deallocation.
ftype.ss, cp0.ss, cmacros.ss, cpnanopass.ss, prims.ss, primdata.ss,
gc.c,
4.ms, root-experr*
- fixed a bug in automatic recompilation handling of missing include
files specified with absolute pathnames or pathnames starting with
"./" or "..": was erroring out in file-modification-time with a
file-not-found or other exception rather than recompiling.
syntax.ss,
7.ms, root-experr*, patch*
- changed inline vector-for-each and string-for-each code to
put the last call to the procedure in tail position, as was
already done for the library definitions and for the inline
code for for-each.
cp0.ss,
5_4.ms, 5_6.ms
- the compiler now generates better inline code for the bytevector
procedure. instead of one byte memory write for each argument,
it writes up to 4 (32-bit machines) or 8 (64-bit machines) bytes
at a time, which almost always results in fewer instructions and
fewer writes.
cpnanopass.ss,
bytevector.ms
- packaged unchanging implicit reader arguments into a single record
to reduce the number of arguments.
read.ss
- recoded run-vector to handle zero-length vectors. it appears
we're not presently generating empty vectors (representing empty
groups), but the fasl format permits them.
7.ss
original commit: 7be1d190de7171f74a1ee71e348d3e6310392686
Add a parameter to constrain the compiler (off by default) to
implement `+`, `*`, and variants as left-associative when given
multiple arguments.
original commit: d126ba3364893e66263c65af1cd6dbdd8b021439
When writing a fasl stream, add extra graph points as needed
to limit recursive reading to about 500 non-tail recursions.
original commit: a6759efdc6ac68e45ec8755a5fe9b75038e173a5
Use `vfasl-convert-file` to convert to the vfasl format, something
like this:
(vfasl-convert-file "orig/petite.boot" "new/petite.boot" '())
(vfasl-convert-file "orig/scheme.boot" "new/scheme.boot" '("petite"))
(vfasl-convert-file "orig/racket.boot" "new/racket.boot" '("petite" "scheme"))
original commit: a40886e2fba741ca8cfc5ebd16b902d6414da0ae
Libraries incorporated via compile-whole-program are, by default,
not visible to environment or eval, unless libs-visible? is true;
complain if we try to visit such libraries.
original commit: 220dca39d0cb482a1cff3f31b8a3197f8b5ee1bc
Removed counter field from prelex, using the operand field instead to
provide the index into the fxmap. This follows other uses within the compiler
where we use the operand field as a little place for state that is used
within a single pass. This has a few advantages. First, it keeps the record a
little smaller. Second, it means that the prelex numbering can start from 0
for each compilation unit, which should help keep the numbers for the fxmap a
bit smaller in longer running sessions with multiple calls to the compiler.
Finally, it avoids adding to the burden of the tc-mutex, since within the pass
it is safe for us to set the prelexes, since only the instance of the pass
holding this block of code has a handle on it. As part of this change
prelex-counter is now defined in cptypes and the operand is cleared after the
variables go out of scope.
base-lang.ss
Fixed the highest-set-bit function in fxmap so that it will work in the 32-bit
versions of Chez Scheme. The fxsrl by 32 raises an exception, and was leading
to tests to fail in 32-bit mode.
fxmap.ss
Restructured predicate-implies? so that it uses committed choice instead of
uncommitted choice in comparing x and y. Basically, this means, instead of
doing:
(or
(and (predicate-1? x) (predicate-1? y) ---)
(and (predicate-2? x) (predicate-2? y) ---)
...)
we now do:
(cond
[(predicate-1? x) (and (predicate-1? y) ---)]
[(predicate-2? x) (and (predicate-2? y) ---)]
...)
This avoids running predicates on x that we know will fail because an earlier
predicate matches, generally getting out of the predicate faster. This did
require a little restructuring, because in some cases x was dominant and in
other cases y was dominant. This is now restructured with y dominate, after
the eq? and x 'bottom check.
Replaced let-values calls with cata-morphism syntax, including removal of maps
that built up a list of values that then needed to be separated out with
(map car ...) (map cadr ...) etc. calls. This avoid building up structures we
don't need, since the nanopass framework will generate a mutltivalued let for
these situations.
The if clause in cptypes/raw now uses types1 (the result of the recursive call
on e1) in place of the incoming types clause when processing the e2 or e3
expressions in the cases where e1 is known statically to produce either a false
or non-false value.
Fixed a bug with directly-applied variable arity lambda. The original code
marked all directly-applied variable arity lambda's as producing bottom,
because it was chacking for the interface to be equal to the number of
arguments. However, variable arity functions are represented with a negative
number. For instance, the original code would transform the expression:
(begin
((lambda (a . b) (set! t (cons* b a t))) 'a 'b 'c)
t)
to
((lambda (a . b) (set! t (cons* b a t))) 'a 'b 'c)
anticipating that the call would raise an error, however, it is a perfectly
valid (if some what unusual) expression. I tried to come up with a test for
this, however, without building something fairly complicated, it is difficult
to get past cp0 without cp0 turning it into something like:
(let ([b (list 'b 'c)])
(set! t (cons* b 'a t))
t)
Fixed make-time, time-second-set!, and time-second to indicate that second can
be an exact-integer, since it is not really restricted to the fixnum range (and
if fact we even test this fact in the mats on 32-bit machines).
primdata.ss
Changed check of prelex-was-assigned (which is not reliably on the input to any
give pass) with prelex-assigned, which should always have an accurate, if
conservative, value in it.
Added enable-type-recovery parameter to allow the type recover to be turned on
and off, and added cptype to the cp0 not run path that runs cpletrec, so that
cptypes can be run independent of cp0. This is helpful for testing and allows
us to benefit from type recovery, even in cases where we do not want cp0 to
perform any inlining.
compile.ss, front.ss, primdata.ss
Stylistic changes, mostly for consistency with other parts of the compiler,
though I'm not married to these changes if you'd really prefer to keep things
the way the are.
1. clauses of define-record type now use parenthesis instead of square brackets.
2. indented by 2 spaces where things were only indented by one space
3. define, let, define-pass, nanopass pass productions clauses, now use
parenthesis for outer markers instead of square brackets.
fxmap.ss,
original commit: 5c6c5a534ff708d4bff23f6fd48fe6726a5c4e05
- fixed an issue with the library system where an exception that occurs
during visit or revisit left the library in an inconsistent state that
caused it to appear that it was still in the process of running. This
manifested in it raising a cyclic dependency exception, even though
there really is not a cyclic dependency. The various library
management functions involved will now reset the part of the library
when an exception occurs. This also means that if the library visit
or revisit failed for a transient reason (such as a missing or
incorrect library version that can be fixed by updating the
library-directories) it is now possible to recover from these errors.
expand-lang.ss, syntax.ss, interpret.ss, compile.ss, cprep.ss,
8.ms
original commit: 6dbd72496fb4eaf5fb65453d0ae0a75f0ef2ad80
commonizatio pass and support for specifying default record
equal and hash procedures:
- more staid and consistent Mf-cross main target
Mf-cross
- cpletrec now replaces the incoming prelexes with new ones so
that it doesn't have to alter the flags on the incoming ones, since
the same expander output is passed through the compiler twice while
compiling a file with macro definitions or libraries. we were
getting away without this just by luck.
cpletrec.ss
- pure? and ivory? now return #t for a primref only if the prim is
declared to be a proc, since some non-proc prims are mutable, e.g.,
$active-threads and $collect-request-pending.
cp0.ss
- $error-handling-mode? and $eol-style? are now properly declared to
be procs rather than system state variables.
primdata.ss
- the new pass $check-prelex-flags verifies that prelex referenced,
multiply-referenced, and assigned flags are set when they
should be. (it doesn't, however, complain if a flag is set
when it need not be.) when the new system parameter
$enable-check-prelex-flags is set, $check-prelex-flags is
called after each major pass that produces Lsrc forms to verify
that the flags are set correctly in the output of the pass.
this parameter is unset by default but set when running the
mats.
cprep.ss, back.ss, compile.ss, primdata.ss,
mats/Mf-base
- removed the unnecessary set of prelex referenced flag from the
build-ref routines when we've just established that it is set.
syntax.ss, compile.ss
- equivalent-expansion? now prints differences to the current output
port to aid in debugging.
mat.ss
- the nanopass that patches calls to library globals into calls to
their local counterparts during whole-program optimization now
creates new prelexes and sets the prelex referenced, multiply
referenced, and assigned flags on the new prelexes rather than
destructively setting flags on the incoming prelexes. The
only known problems this fixes are (1) the multiply referenced
flag was not previously being set for cross-library calls when
it should have been, resulting in overly aggressive inlining
of library exports during whole-program optimization, and (2)
the referenced flag could sometimes be set for library exports
that aren't actually used in the final program, which could
prevent some unreachable code from being eliminated.
compile.ss
- added support for specifying default record-equal and
record-hash procedures.
primdata.ss, cmacros.ss, cpnanopass.ss, prims.ss, newhash.ss,
gc.c,
record.ms
- added missing call to relocate for subset-mode tc field, which
wasn't burning us because the only valid non-false value, the
symbol system, is in the static generation after the initial heap
compaction.
gc.c
- added a lambda-commonization pass that runs after the other
source optimizations, particularly inlining, and a new parameter
that controls how hard it works. the value of commonization-level
ranges from 0 through 9, with 0 disabling commonization and 9
maximizing it. The default value is 0 (disabled). At present,
for non-zero level n, the commonizer attempts to commonize
lambda expressions consisting of 2^(10-n) or more nodes.
commonization of one or more lambda expressions requires that
they have identical structure down to the leaf nodes for quote
expressions, references to unassigned variables, and primitives.
So that various downstream optimizations aren't disabled, there
are some additional restrictions, the most important of which
being that call-position expressions must be identical. The
commonizer works by abstracting the code into a helper that
takes the values of the differing leaf nodes as arguments.
the name of the helper is formed by concatenating the names of
the original procedures, separated by '&', and this is the name
that will show up in a stack trace. The source location will
be that of one of the original procedures. Profiling inhibits
commonization, because commonization requires profile source
locations to be identical.
cpcommonize.ss (new), compile.ss, interpret.ss, cprep.ss,
primdata.ss, s/Mf-base,
mats/Mf-base
- cpletrec now always produces a letrec rather than a let for
single immutable lambda bindings, even when not recursive, for
consistent expand/optimize output whether the commonizer is
run or not.
cpletrec.ss,
record.ms
- trans-make-ftype-pointer no longer generates a call to
$verify-ftype-address if the address expression is a call to
ftype-pointer-address.
ftype.ss
original commit: b6a3dcc814b64faacc9310fec4a4531fb3f18dcd
procedures with large numbers of variables:
- added pass-time tracking for pre-cpnanopass passes to compile.
compile.ss
- added inline handler for fxdiv-and-mod
cp0.ss, primdata.ss
- changed order in which return-point operations are done (adjust
sfp first, then store return values, then restore local saves) to
avoid storing return values to homes beyond the end of the stack
in cases where adjusting sfp might result in a call to dooverflood.
cpnanopass.ss, np-languages.ss
- removed unused {make-,}asm-return-registers bindings
cpnanopass.ss
- corrected the max-fv value field of the lambda produced by the
hand-coded bytevector=? handler.
cpnanopass.ss
- reduced live-pointer and inspector free-variable mask computation
overhead
cpnanopass.ss
- moved regvec cset copies to driver so they aren't copied each
time a uvar is assigned to a register. removed checks for
missing register csets, since registers always have csets.
cpnanopass.ss
- added closure-rep else clause in record-inspector-information!.
cpnanopass.ss
- augmented tree representation with a constant representation
for full trees to reduce the overhead of manipulating trees or
subtress with all bits set.
cpnanopass.ss
- tree-for-each now takes start and end offsets; this cuts the
cost of traversing and applying the action when the range of
applicable offsets is other than 0..tree-size.
cpnanopass.ss
- introduced the notion of poison variables to reduce the cost of
register/frame allocation for procedures with large sets of local
variables. When the number of local variables exceeds a given
limit (currently hardwired to 1000), each variable with a large
live range is considered poison. A reasonable set of variables
with large live ranges (the set of poison variables) is computed
by successive approximation to avoid excessive overhead. Poison
variables directly conflict with all spillables, and all non-poison
spillables indirectly conflict with all poison spillables through
a shared poison-cset. Thus poison variables cannot live in the
same location as any other variable, i.e., they poison the location.
Conflicts between frame locations and poison variables are handled
normally, which allows poison variables to be assigned to
move-related frame homes. Poison variables are spilled prior to
register allocation, so conflicts between registers and poison
variables are not represented. move relations between poison
variables and frame variables are recorded as usual, but other
move relations involving poison variables are not recorded.
cpnanopass.ss, np-languages.ss
- changed the way a uvar's degree is decremented by remove-victim!.
instead of checking for a conflict between each pair of victim
and keeper and decrementing when the conflict is found, remove-victim!
now decrements the degree of each var in each victim's conflict
set. while this might decrement other victims' degrees unnecessarily,
it can be much less expensive when large numbers of variables are
involved, since the number of conflicts between two non-poison
variables should be small due to the selection process for
(non-)poison variables and the fact that the unspillables introduced
by instruction selection should also have few conflicts. That
is, it reduces the worst-case complexity of decrementing degrees
from O(n^2) to O(n).
cpnanopass.ss
- took advice in compute-degree! comment to increment the uvars in
each registers csets rather than looping over the registers for
each uvar asking whether the register conflicts with the uvar.
cpnanopass.ss
- assign-new-frame! now zeros out save-weight for local saves, since
once they are explicitly saved and restored, they are no longer
call-live and thus have no save cost.
cpnanopass.ss
- desensitized the let-values source-caching timing test slightly
8.ms
- updated allx, bullyx patches
patch*
original commit: 3a49d0193ae57b8e31ec6a00b5b49db31a52373f
Add optional beginning-line and beginning-column components to a
source object, so that line and column information can be recorded
independent of the file. Add `locate-source-object-source` to use
the recorded information. Add a cache for `locate-source` as enabled by
the `use-cache?` optional argument, which can avoid compilation times
that are quadratic in the number of `let-values` or `define-values`
forms.
original commit: b36fab81d5041a54ce01a422395eee79d2f930bc
line from the wpo file (if it has one) to the object file.
compile.ss,
7.ms
- stex is now a submodule. csug/Makefile and release_notes/Makefile
set and use the required Scheme and STEXLIB variables accordingly.
they default the machine type to a6le, but this can be overridden
and is by the generated top-level Makefile. the generated top-level
Makefile now has a new docs target that runs make in both csug and
release_notes, and an updated distclean target that cleans the same.
the annoying csug Makefile .fig.pdf rule redefinition is now gone.
copyright.stex and csug.stex now list May 2016 as the revision month
and date; this will have to be updated for future releases.
configure, makefiles/Makefile.in,
csug/Makefile, copyright.stex, csug.stex,
release_notes/Makefile
- rebuilt the boot files
original commit: 4bd78a692dd4ca2f88af5d404fd0993a2d141e7b
