added fix for whole program/library compilation bug with help from
@owaddell who originally reported the problem in issue 386. this bug
arises from the way the parts of the combined library, and their
binary dependencies, are invoked when one of the constituent libraries
is invoked. consider, for example, a combined library that contains
(A) and (B), where (B) depends on a binary library (C). depending on
the sort order of (A) and (B), which may be unconstrained in the
partial ordering established by library dependencies, invoking (A) may
result in the invoke code for (B) being run first, without (B) ever
being explicitly invoked. this can result in bindings required from
(C) by the invoke code in (B) to be unbound. even in the case where
(A) comes before (B) in the topological sort, if they are combined
into the same cluster, (B)'s invoke code will be run as part of
invoking (A). the solution is two part: first we extend the invoke
requirements of the first library in the cluster to include the binary
libraries that precede it in the topological sort and add a dependency
on the first library in the cluster to all of the other libraries in
the cluster. this means no matter which library in the cluster is
invoked first, it will cause the first library to be invoked, in turn
ensuring the binary libraries that precede it are invoked. when there
are multiple clusters, a dependency is added from each cluster to the
first library in the cluster that precedes it. this ensures that
invoking a library in a later cluster first, will still cause all of
the dependencies of the previous clusters to be invoked. ultimately,
these extra dependencies enforce an ordering on the invocation of the
source and binary libraries that matches the topological sort, even if
the topological sort was under constrained. to maintain the property
that import requirements are a superset of the invoke and visit
requirements, we also extend the import requirements to include the
extended invoke requirements. the import requirements are also added
to the dependency graph to further constrain the topological sort and
ensure that we do not introduce artificial cycles in the import graph.
compile.ss,
7.ms,
root-experr*, patch*
original commit: 09bba001a33a5ee9268f1e5cf0cc118e8a2eec7f
exposed the default-library-search-handler and a library-search-handler
parameter to permit more control over the search for libraries during
import, compile-whole-library, and compile-whole-program
original commit: 7b4fdd374f9cb973de1143bfcc830194b36befda
on p to account for the procedure? check at optimize-level 2.
cpletrec.ss
- fixed bug in check-prelex-flags: was hardwiring $cpcheck-prelex-flags
"after" argument to 'uncprep rather than passing along its argument.
compile.ss
- commented out local definition of sorry! so that problems detected
by $cpcheck-prelex-flags actually result in a raised exception.
cprep.ss
original commit: 674e2180d6893000446590038dcacf0ef661e564
InstallPrefix, since it didn't and can't play well with the
generated config.h. Instead removed InstallPrefix entirely so
it isn't an attractive hazzard.
configure, makefiles/Mf-install.in
original commit: 21c8b40bbe4431795824e48042112820872a1fe5
controversial, unless I damaged something in the process of integrating
them with other recent changes. the user's guide and release notes
have been updated as well to reflect the changes of interest to end
users.
- the body of load-library is now wrapped in a $pass-time with
to show the time spent loading libraries separately from the time
spent in expand.
syntax.ss
- interpret now plays the pass-time game
interpret.ss
- added compile-time-value? predicate and
compile-time-value-value accessor
syntax.ss, primdata.ss,
8.ms, primvars.ms, root-experr*
- $pass-stats now returns accurrate stats for the currently timed
pass.
7.ss
- compile-whole-program and compile-whole-library now propagate
recompile info from the named wpo file to the object file
to support maybe-compile-program and maybe-compile-library in
the case where compile-whole-{program,library} overwrites the
original object file.
compile.ss,
7.ms, mat.ss, primvars.ms
- replaced the ancient and unusable bintar with one that creates
a useful tarball for binary installs
bintar
- generated Mf-install InstallBin (InstallLib, InstallMan) now
correctly indirects through InstallPrefix if the --installbin
(--installlib, --installman) configure flag is not present.
src/configure
- removed definition of generate-procedure-source-information
patch.ss
- guardian tconc cells are now allocated in generation 0 in the hope
that they can be released more quickly.
gc.c
- added ftype-guardian syntax: (ftype-guardian A) creates a new
guardian for ftype pointers of type A, the first base field (or
one of the first base fields in the case of unions) of which must
be a word-sized integer with native endianness representing a
reference count. ftype pointers are registered with and retrieved
from the guardian just like objects are registered with and
retrieved from any guardian. the difference is that the garbage
collector decrements the reference count before resurrecting an
ftype pointer and resurrects only those whose reference counts
become zero, i.e., are ready for deallocation.
ftype.ss, cp0.ss, cmacros.ss, cpnanopass.ss, prims.ss, primdata.ss,
gc.c,
4.ms, root-experr*
- fixed a bug in automatic recompilation handling of missing include
files specified with absolute pathnames or pathnames starting with
"./" or "..": was erroring out in file-modification-time with a
file-not-found or other exception rather than recompiling.
syntax.ss,
7.ms, root-experr*, patch*
- changed inline vector-for-each and string-for-each code to
put the last call to the procedure in tail position, as was
already done for the library definitions and for the inline
code for for-each.
cp0.ss,
5_4.ms, 5_6.ms
- the compiler now generates better inline code for the bytevector
procedure. instead of one byte memory write for each argument,
it writes up to 4 (32-bit machines) or 8 (64-bit machines) bytes
at a time, which almost always results in fewer instructions and
fewer writes.
cpnanopass.ss,
bytevector.ms
- packaged unchanging implicit reader arguments into a single record
to reduce the number of arguments.
read.ss
- recoded run-vector to handle zero-length vectors. it appears
we're not presently generating empty vectors (representing empty
groups), but the fasl format permits them.
7.ss
original commit: 7be1d190de7171f74a1ee71e348d3e6310392686
- fixed a bug in which instantiating a static foreign-callable code object
fails with an invalid memory reference because the collector has
discarded its relocation information. foreign-callable code objects
are now flagged as "templates", and the collector now refuses to
discard relocation information for code objects marked as templates
when copying them to the static generation.
cmacros.ss, cpnanopass.ss,
gc.c,
7.ms
- committing updated boot/*/equates.h (without the boot files, which are
still usable for bootstrapping)
boot/*/*.h
- updated release notes
release_notes.stex
original commit: 71d3abba684e04b134720ea1bd9a8c847c38ac5f
fix#389 (apply doesn't throw exception when last argument isn't a list)
fold-primref and fold-primref2 ignored app-convention when
attempting to fold certain primitive calls in 'test and 'effect
context and when falling back on the default primitive handler.
We now residualize primitive references if the app-convention
is not 'call. The original fold-primref2 already bypassed the
inline handler when the app-convention was not 'call.
original commit: f9d10c4cf2e6cd184ad7429f251360a738600959
The `dofretu...*` intrinsics used %ac1 without declaring it as a used
registers, which effectively broke register allocation for handling
string/bytevector foreign-call results or callable arguments.
original commit: 993fb9036acad5445319f458fd971b1a1d8e9f84
reclaimed by the collector and must be released explicitly by the
programmer via (profile-release-counters).
pdhtml.ss, primdata.ss,
globals.h, externs.h, fasl.c, prim5.c, prim.c, alloc.c, scheme.c,
misc.ms,
release_notes.stex, system.stex
original commit: 68e20f721618dbaf4c1634067c2bee24a493a750
Also, for completeness, correct the listing of callee-save registers
in callable return for x86 & x86_64.
original commit: 4cd942be6ab2eb5e02f6d6c5c509db3131bd015f
- when thread_get_room exhausts the local allocation area, it now
goes through a common path with S_get_more_room to allocate a new
local allocation area when appropriate. this can greatly reduce
the use of global allocation (and the number of tc mutex acquires
in threaded builds) when a lot of small objects are allocated by
C code with no intervening Scheme-side allocation or dirty writes.
alloc.c, types.h, externs.h
original commit: 93dfa7674a95837e5a22bc622fecc50b0224f60d
- the compiler now uses a temporary variable to hold the return
value of a nested call across the restore-local-saves form to
properly handle the case where the destination lvalue is an mref
whose base and/or index is a local save.
cpnanopass.ss,
misc.ms
original commit: 835cbc2430be4f7381cee27133d42e77ace2b37f
bootstrap failures after small changes like the recent change to
procedure names, so we don't have to rebuild the boot files as often.
Mf-base
original commit: 94aaebe54807780791b8373017c979f59e6c45f1
In some cases, such as
(define g (let ([f (lambda (x) x)]) f))
the inner name `f` could get replaced by the outer name `g`.
Prefer the inner name so that macros can use a `let` wrapper
to reliably name a procedure.
original commit: 9ae0f856b0f81610c6caea13a5bfdf3b135de5b0
Libraries incorporated via compile-whole-program are, by default,
not visible to environment or eval, unless libs-visible? is true;
complain if we try to visit such libraries.
original commit: 220dca39d0cb482a1cff3f31b8a3197f8b5ee1bc
Improve error reporting and improve docs as suggested by Andy, and
adjust `conv` -> `conv*` to fit a naming convention.
original commit: b34817aea5d3c4862e7bb313ee9f5281472a832f
Embarrassingly, I committed this change on the wrong branch initially.
Added -Wno-implicit-fallthrough flag to macOS C makefiles.
c/Mf-a6osx, c/Mf-i3osx, c/Mf-ta6osx, c/Mf-ti3osx
original commit: 8eb8336a7d2870f8e592f060bab8321703e40b48
- fixed an issue with the library system where an exception that occurs
during visit or revisit left the library in an inconsistent state that
caused it to appear that it was still in the process of running. This
manifested in it raising a cyclic dependency exception, even though
there really is not a cyclic dependency. The various library
management functions involved will now reset the part of the library
when an exception occurs. This also means that if the library visit
or revisit failed for a transient reason (such as a missing or
incorrect library version that can be fixed by updating the
library-directories) it is now possible to recover from these errors.
expand-lang.ss, syntax.ss, interpret.ss, compile.ss, cprep.ss,
8.ms
original commit: 6dbd72496fb4eaf5fb65453d0ae0a75f0ef2ad80
the signal handler could trip over the NULL jumpbuf in a CCHAIN record.
schlib.c
remade boot files
original commit: d8c270403121547101cb523cc1f80a569dbb0378
So the reduced version checks that p is a procedure
Also make the same change for #2%for-each.
cp0.ss, 4.ms
original commit: 5caa11c85bc74c0af25ac215d48b7f5f0c1d3e42
commonizatio pass and support for specifying default record
equal and hash procedures:
- more staid and consistent Mf-cross main target
Mf-cross
- cpletrec now replaces the incoming prelexes with new ones so
that it doesn't have to alter the flags on the incoming ones, since
the same expander output is passed through the compiler twice while
compiling a file with macro definitions or libraries. we were
getting away without this just by luck.
cpletrec.ss
- pure? and ivory? now return #t for a primref only if the prim is
declared to be a proc, since some non-proc prims are mutable, e.g.,
$active-threads and $collect-request-pending.
cp0.ss
- $error-handling-mode? and $eol-style? are now properly declared to
be procs rather than system state variables.
primdata.ss
- the new pass $check-prelex-flags verifies that prelex referenced,
multiply-referenced, and assigned flags are set when they
should be. (it doesn't, however, complain if a flag is set
when it need not be.) when the new system parameter
$enable-check-prelex-flags is set, $check-prelex-flags is
called after each major pass that produces Lsrc forms to verify
that the flags are set correctly in the output of the pass.
this parameter is unset by default but set when running the
mats.
cprep.ss, back.ss, compile.ss, primdata.ss,
mats/Mf-base
- removed the unnecessary set of prelex referenced flag from the
build-ref routines when we've just established that it is set.
syntax.ss, compile.ss
- equivalent-expansion? now prints differences to the current output
port to aid in debugging.
mat.ss
- the nanopass that patches calls to library globals into calls to
their local counterparts during whole-program optimization now
creates new prelexes and sets the prelex referenced, multiply
referenced, and assigned flags on the new prelexes rather than
destructively setting flags on the incoming prelexes. The
only known problems this fixes are (1) the multiply referenced
flag was not previously being set for cross-library calls when
it should have been, resulting in overly aggressive inlining
of library exports during whole-program optimization, and (2)
the referenced flag could sometimes be set for library exports
that aren't actually used in the final program, which could
prevent some unreachable code from being eliminated.
compile.ss
- added support for specifying default record-equal and
record-hash procedures.
primdata.ss, cmacros.ss, cpnanopass.ss, prims.ss, newhash.ss,
gc.c,
record.ms
- added missing call to relocate for subset-mode tc field, which
wasn't burning us because the only valid non-false value, the
symbol system, is in the static generation after the initial heap
compaction.
gc.c
- added a lambda-commonization pass that runs after the other
source optimizations, particularly inlining, and a new parameter
that controls how hard it works. the value of commonization-level
ranges from 0 through 9, with 0 disabling commonization and 9
maximizing it. The default value is 0 (disabled). At present,
for non-zero level n, the commonizer attempts to commonize
lambda expressions consisting of 2^(10-n) or more nodes.
commonization of one or more lambda expressions requires that
they have identical structure down to the leaf nodes for quote
expressions, references to unassigned variables, and primitives.
So that various downstream optimizations aren't disabled, there
are some additional restrictions, the most important of which
being that call-position expressions must be identical. The
commonizer works by abstracting the code into a helper that
takes the values of the differing leaf nodes as arguments.
the name of the helper is formed by concatenating the names of
the original procedures, separated by '&', and this is the name
that will show up in a stack trace. The source location will
be that of one of the original procedures. Profiling inhibits
commonization, because commonization requires profile source
locations to be identical.
cpcommonize.ss (new), compile.ss, interpret.ss, cprep.ss,
primdata.ss, s/Mf-base,
mats/Mf-base
- cpletrec now always produces a letrec rather than a let for
single immutable lambda bindings, even when not recursive, for
consistent expand/optimize output whether the commonizer is
run or not.
cpletrec.ss,
record.ms
- trans-make-ftype-pointer no longer generates a call to
$verify-ftype-address if the address expression is a call to
ftype-pointer-address.
ftype.ss
original commit: b6a3dcc814b64faacc9310fec4a4531fb3f18dcd
a non-procedure exception when the first argument is not a procedure, even
when the rest of the program is compiled at optimize level 3.
4.ms, root-experr-compile-0-f-f-f, patch-compile-0-t-f-f,
patch-compile-0-f-t-f, patch-interpret-0-f-t-f, patch-interpret-0-f-f-f,
patch-interpret-3-f-t-f, patch-interpret-3-f-f-f
original commit: 7916447d1a482ec91ae63927692053d727d9b459
- fixed substring-fill! and vector-fill! to return void, reflecting the
documented return value of unspecified value. Also changes substring-fill!
to use define-who instead of repeating 'substring-fill! in all the error
messages.
5_4.ss, 5_6.ss
original commit: 3f65788b5422693f3648a9e2fe575f464eb31ccd
compilation now treat a malformed object file as if it were
not present and needs to be regenerated. A malformed object
file (particularly a truncated one) might occur if the compiling
processes is killed or aborts before it has a chance to delete
a partial object file.
syntax.ss,
7.ms
original commit: c2cb8c79a925c0eb2f9d589e3a497712800bd1dc
entries x and y in the list produced by the sort call, if x's
bfp = y's bfp, x should come before y if x's efp < y's efp.
The idea is that enclosing entries should always come later
in the list. this affects only languages where two expressions
can start at the same character position.
pdhtml.ss
expanded capability of ez-grammar with support for simpl
parsing of binary operators w/precedence and associativity
and automatically generated markdown grammar descriptions.
ez-grammar-test.ss now also doubles as a test of pdhtml for
algebraic languages.
mats/examples.ms,
examples/ez-grammar.ss, examples/ez-grammar-test.ss,
examples/Makefile
original commit: 53b8d16a1e86f3956585dbec0c7b573e485f7844
test and ephemeron gc test while resensitizing the former
8.ms, 4.ms
various formatting and comment corrections
workarea,
s/Mf-base, bytevector.ss, cpnanopass.ss, date.ss,
5_6.ms, examples.ms
original commit: 19e2505fc6477fce2d1d0e61187bd504b58ea994
procedures with large numbers of variables:
- added pass-time tracking for pre-cpnanopass passes to compile.
compile.ss
- added inline handler for fxdiv-and-mod
cp0.ss, primdata.ss
- changed order in which return-point operations are done (adjust
sfp first, then store return values, then restore local saves) to
avoid storing return values to homes beyond the end of the stack
in cases where adjusting sfp might result in a call to dooverflood.
cpnanopass.ss, np-languages.ss
- removed unused {make-,}asm-return-registers bindings
cpnanopass.ss
- corrected the max-fv value field of the lambda produced by the
hand-coded bytevector=? handler.
cpnanopass.ss
- reduced live-pointer and inspector free-variable mask computation
overhead
cpnanopass.ss
- moved regvec cset copies to driver so they aren't copied each
time a uvar is assigned to a register. removed checks for
missing register csets, since registers always have csets.
cpnanopass.ss
- added closure-rep else clause in record-inspector-information!.
cpnanopass.ss
- augmented tree representation with a constant representation
for full trees to reduce the overhead of manipulating trees or
subtress with all bits set.
cpnanopass.ss
- tree-for-each now takes start and end offsets; this cuts the
cost of traversing and applying the action when the range of
applicable offsets is other than 0..tree-size.
cpnanopass.ss
- introduced the notion of poison variables to reduce the cost of
register/frame allocation for procedures with large sets of local
variables. When the number of local variables exceeds a given
limit (currently hardwired to 1000), each variable with a large
live range is considered poison. A reasonable set of variables
with large live ranges (the set of poison variables) is computed
by successive approximation to avoid excessive overhead. Poison
variables directly conflict with all spillables, and all non-poison
spillables indirectly conflict with all poison spillables through
a shared poison-cset. Thus poison variables cannot live in the
same location as any other variable, i.e., they poison the location.
Conflicts between frame locations and poison variables are handled
normally, which allows poison variables to be assigned to
move-related frame homes. Poison variables are spilled prior to
register allocation, so conflicts between registers and poison
variables are not represented. move relations between poison
variables and frame variables are recorded as usual, but other
move relations involving poison variables are not recorded.
cpnanopass.ss, np-languages.ss
- changed the way a uvar's degree is decremented by remove-victim!.
instead of checking for a conflict between each pair of victim
and keeper and decrementing when the conflict is found, remove-victim!
now decrements the degree of each var in each victim's conflict
set. while this might decrement other victims' degrees unnecessarily,
it can be much less expensive when large numbers of variables are
involved, since the number of conflicts between two non-poison
variables should be small due to the selection process for
(non-)poison variables and the fact that the unspillables introduced
by instruction selection should also have few conflicts. That
is, it reduces the worst-case complexity of decrementing degrees
from O(n^2) to O(n).
cpnanopass.ss
- took advice in compute-degree! comment to increment the uvars in
each registers csets rather than looping over the registers for
each uvar asking whether the register conflicts with the uvar.
cpnanopass.ss
- assign-new-frame! now zeros out save-weight for local saves, since
once they are explicitly saved and restored, they are no longer
call-live and thus have no save cost.
cpnanopass.ss
- desensitized the let-values source-caching timing test slightly
8.ms
- updated allx, bullyx patches
patch*
original commit: 3a49d0193ae57b8e31ec6a00b5b49db31a52373f
month and year at the time of generation.
csug.stex, copyright.stex
- Updated configuration to set machine types in the CSUG and release notes
make files, and updated distclean target to remove these files.
configure, makefiles/Makefile-csug.in (renamed from csug/Makefile),
makefiles/Makefile-release_notes.in
(renamed from release_notes/Makefile),
makefiles/Makefile
original commit: d4fdb3e0c88c40cceeeeeb52032068408edc7a6e
redundant keys and expands into exclusive-cond rather than cond.
it catches references to => before expanding into exclusive-cond
to avoid supporting => as an undocumented and useless extension
of the case syntax. the r6rs:case and case macros now require
multiple clauses rather than leaving the enforcement to exclusive-cond,
and the exclusive-cond macro now requires multiple clauses rather
than leaving the enforcement to cond.
syntax.ss,
4.ms, root-experr*, patch*
original commit: 303921d8515b101c558a056dcf9c05f7cad97f4a
an existing setting will not interfere with the build process, and
added a note to BUILDING that CHEZSCHEMELIBDIRS should be unset in
Version 9.5 and before.
s/Mf-base, mats/Mf-base, BUILDING
original commit: 1cccd56693911a38ece784da677e35f724950700
