For example, if the result of `(when (box? x) (unbox x))` is not used,
then the `(unbox x)` still must be done, because the box might be an
impersonator. In contrast, `(when (box*? x) (unbox* x))` can be
dropped, since `unbox*` is an authentic unbox.
This change applies to unsafe operations like `unsafe-struct-ref`,
too, and applies to struct accessors for non-authentic structure
types.
Racket CS already preserves operations appropriately.
Relevant to #3487
To make room in the type encoding, remove immutable fxvectors from
Chez Scheme --- which had been added just to go along with immutable
strings, vectors, and bytevectors, but immutable fxvectors do not seem
useful, and they have no counterpart in Racket.
Since the reader now treats #\uFEFF as whitespace, adjust the printer to
escape \uFEFFa.
Thanks, Xsmith!
Writing an actual #\uFEFF character seems bad, even escaped, but
Racket's symbol syntax doesn't have a kind of escaping that uses
different characters than the one to be represented.
Closes#3486
Streamline rktio byte-result copying (main improvement), use fixnum
arithmetic more consistently (minor improvement), and change
`in-bytes`, etc., to avoid some checks in unsafe mode (intermediate
improvement).
When bytes within a Windows path cannot be converted using
`bytes->string/locale` (i.e., when the bytes do not fit a UTF-8
encoding), then leave the bytes alone, instead of triggering a failure
from `bytes->string/locale`.
Fixing this bug uncovered others: `string-locale-downcase` did not
work on an empty byte string on a little-endian machine, and
`in-bytes` and similar reported range errors in terms of "vectors".
Change the way boot images are sent to Chez Scheme by the Racket CS
wrapper, especially in the case where boot images are embedded in an
executable (which is always true for a distriution build). The revised
approach avoids a little filesystem work, and it may help Chez Scheme
pull bytes in faster.
Builing Racket CS on a 64-bit platform requires a little more than 1.5
GB of memory due to whole-program optimization of the Racket core
immplementation. Add a `--disable-wpo` configure option, which keeps
memory use below 0.5 MB to provide the option of building Racket in a
more constrained environment.
Instead of selecting flags through a mixture of `configure` and
"c/Mf-*", determine defaults in `configure`. If `CFLAGS` is provided,
then the defaults are not used. Flags needed for threaded mode are an
exception, and `LDFLAGS` and `LIBS` are expanded even if supplied ---
unless `--disable-auto-flags` is specified.
Closes#3467
The `--sync-docs-only` flag is intended for contexts like pkg-build as
a backstop against unintended and time-consuming activity when the
intent is to assemble documentation.
While sending an absolute path to a foreign library is usually the
right idea to ensure that it's relative to `(current-directory)`, the
`_path` FFI type should not do that automatically --- because
sometimes it's useful to send a relative path to a foreign library,
but most because it hasn't been defined that way in BC.
In building up conversion tables, information from "SpecialCasing.txt"
data was incorrectly merged with data from "UnicodeData.txt" so that
not-quite-so-special casings were fumbled. For Unicode 7.0, the bug
turned out to affect only string downcasing of U+1E9E.
Closes#3475
Thanks to Xsmith!
Changes to use xlocale fixed problems with places and locale settings,
but it caused the initial process-wide locale to stay with the C
locale, which is bad for things like libedit. To get the good part of
the old behavior bback, set the process-wide locale to "" on startup.
When an exact (non-integer) rational is reconstructed from a place
message, normalization could involve bignum operations --- and those
operations might use a cache, but the cache should not refer to the
pages that are specific to a place message being constructed.
Normalization isn't necessary, since the parts were already in a
rational, so the repair is just it skip it.
Relevant to #3456
A table of subprocess handles to finalize was not place-local as it
should have been. The same was true of a table of resolved IP
addresses to finalize.
Related to #3456
The Racket variant of Chez Scheme includes special treatment of deeply
nested structures to avoid a C-stack overflow on unfasl, but the
relevant callbacks had gotten mangled.
Closes#3454
When a would-be loop is called in a would-be loop that turns out not
to be a loop due to an intervening non-loop layer, the outer would-be
loop was not detected as a non-loop.
When the mutability decision on a variable is delayed, but then the
variable is discovered to be mutable before the delay is triggered,
then mutability information could get lost.
Mutability analysis may determine that a `set!` is dead code, and then
the mutated variable might be optimized away, so don't leave the dead
`set!` behind.
When the srcloc-enriched S-expression representation a function that
is available for cross-module inlining has a source that is not a
path, string, or symbol, then the source has to be dropped in the
module's serialized form.
On a 64-bit machine, the problem was a `<` versus `<=`.
On a 32-bit machine where 4294967087 is not a fixnum, the
problem was in how bignums are handled.
Previously, `error` would raise an `exn:fail` with the following message,
because it treated the first of the three strings as a format string
and the other two as its arguments:
> error: format string requires 0 arguments, given 2;
> arguments were:
> " is a procedure, to always escape when called"
> " (by calling raise-blame-error with the arguments it was given"`
The main change is to disable forced GCs during the documentation
phase in a parallel `raco setup`. Since each GC is global (instead of
place local) in Racket CS, these forced GCs created a scaling problem.
Furthermore, they're not really necessary; peak memory use tends to
remain the same without them in a parallel build, since the
".zo"-building phase doens't have extra collections and tends to
establish peak memory use.
A non-parallel build still includes explicitly forced GCs. In that
mode, the cost is modest while reducing peak use by 30%.
A new environment variable appends timestamps (in process time since
startup) for each status line, which is useful for generating more
detailed build plots.
Since a GC with accounting doesn't run in parallel, it discards
owner-thread information (ironically) that is used for parallel
collection. Losing ownership information could reduce parallelism in
later collections --- but only if they run without accounting.
This commit does not turn on ownership tracking, however, because
experiments suggest that it isn't worthwhile for Racket:
* Preserving ownership information slows down a major collection in
accounting mode by 5-10%.
* Racket uses accounting mode only for full collections, and the loss
of ownership information would mostly affect only later full
collections. Since accounting momde tends to stick around, very
little parallelism is actually lost.
* Without further work, accounting mode cannot benefit from
parallelization, because it works by stepping sequentially through
accounting domains, and that aligns with thread ownership. A
possible improvement is to look ahead in the accounting-domain
object sequence to find one that starts in another ownership
domain, and then constrain concurrent tracing from that object to
keep the count private and not leave the ownership domain until the
accounting sequence has caught up.
Besides making it easier to turn on "parallel" collection for
accounting mode, the small refactorings here make it easier to bring
more things, like guardian and weak-pair handling, into the parallel
part of a collection. So, the changes are probably worth keeping for
that purpose.
The collector tries to use roughly the same amount of parallelism as
the number of active threads, but make sure that it doesn't fall back
to an ownership-mangling non-parallel collection if all but one place
happens to be stalled.
Make the exchange of objects by parallel sweepers --- which is needed
when an object being swept by one thread refers to an object owner by
another thread --- much simpler and slightly faster.
The original design of exchanging memory regions didn't really work
out, and it had degrenerated to essentially exchanging objects.
Explicitly exchanging objects instead of regions makes the GC simpler,
because received objects can be just added to the regular sweep stack,
and less additional space<->type correspondence is needed.
Fix mishandling of an expanded `if` where the test position is a
syntax object. The expander's compiler pass from expanded objects to
linkets knows that the syntax object isn't useful, but it tried to be
helpful by preserving the syntax object's content as quoted --- and
that content turns out not to be available, so the syntax object was
replaced by #f, instead.
Closes#3436
Rename definitions that are not exported and that are not used as
inferred function names. The rename is based on a hash of the
right-hand side, instead of being just the position of the definition
in the module, and that should help further reduce diffs in schemified
output after small changes to the source.
Instead of one big lock, have one big lock and one small lock for just
manipulating allocation state. This separation better reflects the
locking needs of parallel collection, where the main collecting thread
holds the one big lock, and helper threads need to take a different
lock while changing allocation state.
This commit also includes changes to cooperate better with LLVM's
thread sanitizer. It doesn't work for parallel collection, but it
seems to work otherwise and helped expose a missing lock and a
questionable use of a global variable.
This ensures that the moveq instruction is inside an IT block.
Backward compatible change - still builds on ARMv6 with Thumb.
This breakage is generally not noticeable on a RPi3 because, although the
CPU is ARMv7, it is generally running an ARMv6 OS.
Disable incremental promotion when backreferences are enabled,
otherwise the backreference list for a generation can have
references to younger-generation objects.
Casting a ptr to an integer of a different size causes a warning - this is only noticeable on 32bits. Fix this by using `TO_VOIDP` and `TO_PTR` helper macros.
This PR fixes two issues regarding disappeared-use in reqprov.rkt
1. Copy/preserve disappeared-use for all provide subforms instead
of only the first one. The following program doesn't have an arrow from
`struct-out` to `#lang racket` prior this PR, but it does after this PR.
#lang racket
(provide a? (struct-out a))
(struct a ())
2. Fix incorrect `disappeared-use` for `struct-out`: it is inappropriate
to check if an identifier is bound to a struct info in provide
pre-transformer because it would not be able to handle backlinks.
This PR moves the attachment from provide pre-transformer to provide
transformer, allowing Check Syntax to draw an arrow for the identifier
`abc` in:
#lang racket
(provide (struct-out abc))
(struct abc ())
Small fixes to enable LTO compilation
To do this in racket/src:
./configure CFLAGS="-O3 -march=native -flto"
LDFLAGS="-O3 -march=native -flto"
--prefix=...
One fix deals with variable `errnum` that might be used uninitialized.
The other is to ensure that `boot_file_data` is marked as used
otherwise, it is removed by LTO and ends up crashing build
in `embed-boot.rkt`.
Instead of having leftover Scheme threads swept by the main thread,
distribute Scheme threads among sweeper threads, and swap into each
thread's content to preserve its allocation ownership. This change
makes parallelism more consistent for different timings that end up
assigning sweepers differently.
Move per-thread allocation information to a separate object, so the
needs of the collector are better separated from the thread
representation. This refactoring sets up a change in the collector to
detangle sweeper threads from Scheme threads.
Change the internal parallelism strategy for the GC to record an owner
for each allocated segment of memory, and have the owner be solely
responsible for copying or marking objects of the segment. When
sweeping, a collecting thread handles references to objects that it
owns or that have been copied or marked already, and it asks another
collecting thread to resweep an object that refers to objects owned by
that that thread. At worst, an object ends up being swept by all
collecting threads, one at a time, but that's unlikely for a given
object.
The approach seems likely to scale better than a lock-based approach,
even the one that used a lightweight, CAS-based lock and retries on
lock failure.
That is, use pairs on the property in more places, as the pair
already was computed and the value-blame function already does
the needful when it sees a pair on the property.
When the GC needs to copy/mark a record, it previously forced a
copy/mark on the record's type descriptor, since the copy/mark needs
information from the descriptor. But the needed information is not in
danger of being overwritten for forwading (since it's after the first
two words of the type descriptor), so it's ok to use the old reference
as-is --- at least in non-counting mode. Simplifying record-type
handling and deferring the record-type update to the sweep phase, the
same as for other components of the record type, makes the GC slightly
faster.
Fall back to `current-directory` when `current-load-relative-directory`
is #f.
This change also affects `deserialize` --- not because byte code
loading uses it directly in this case, but because they share a helper
function, which exposes the issue. This implementation change is
worrying (even though it makes the implementation match the
documentation), but unless we discover that some use of serialization
needs absolute paths deseialized as relative, is seems better to be
consistent everywhere about falling back to `currenrt-directory`. This
aspect of the change can be reverted separately (by adding more code)
if needed.
Closesracket/drracket#421
All allocation is now thread-local, which recovers a small bit of
performance that was lost when adding thread-local allocation
alongside global allocation.
Parallelism uses thread contexts created by Chez Scheme threads (which
correspond to Racket places and future-running threads), but it
creates its own OS-level threads to perform collection. The number of
collection-helper threads is limited to the number of active Chez
Scheme threads. Only the main "sweep" pass runs in parallel --- that
is, after roots have been traversed, and before weak references and
finalization are handled --- but that's the bulk of collection work.
Also, memory-accounting collections always run as single-threaded.
Improved support for thread-location allocation (and using more
fine-grained locks in fasl reading) may provide a small direct
benefit, but the change is mainly intended as setup for more
parallelism in the collector.
There's a trade-off between keeping the distribution sizes small and
making ".boot" files available for convenient embedding, even though
embedding is relatively rare. For Unix platforms, since you have to
build from source to get a static library for embedding anyway, we'll
leave out ".boot" files. For Mac OS, the distribution's "Racket"
framework includes ".boot" files --- even though the framework is
itself unused for a normal distribution build, since signing and
notarization are handled by embedded the boot files in an executable,
but the framework was kept for a kind of backward compatibility. For
Windows, the Racket DLL can be used for embedding, so the ".boot"
files would be the only missing piece; also, they were already
included in a cross-built distribution.
Update "Inside" to note that ".boot" files must be built on Unix and
to clarify the location of ".boot" files on Mac OS.
Closes#3377
The A32 instruction set has an interesting encoding of immediate
values where a larger value sometimes fits in a smaller set of
instructions. That turns out to be a bad property for loading a return
address, because it means that the as label computations push code
further away, a contracting return-address calculation can pull code
back nearer, and this push-and-pull can keep the label allocator from
arriving at a fixpoint.
This became a bigger problem with 8834597c1f, which creates
return-label references that go backwards and where the offset can be
much larger than the normal, forward references.
See #3378: The possibility of mutation should be considered in
`:do-in` in somme rare cases, while it's not clear that there's
anything better to be done for mutation of the list accumulators in
`for/lists`. At least make the pitfalls clearer in the documentation.
This commit doesn't update nanopass itself, but adapts `rktboot`
so it can be used with the main Chez Scheme bbranch. It also
adjust "cpnanopass.ss" to avoid different behavior between the
old and newer versions of nanopass.
The Scheme stack pointer was left in call state when the number of
results is wrong, with the intent of exposing the enclosing function's
frame for debugging purposes, but there's no guanrantee that the
result address is still on the stack (i.e., the continuation make have
been captured). Reinstall the return address before calling the
exception handler.
Avoid coercing an integer to a flonum when doing so loses
precision. It's especially helpfu lto preserve oddness
versus evenness.
Closes#3360Closes#3100
Experiment with removing built-in Racket functions in stack traces to
make the trace less noisy. "Built-in" is defined as code that exists
in the built-in modules. On CS, built-in code is detected as residing
in the static generation. (Also, on CS, the code must have a name but
no source location or detailed debugging information to be
suppressed.) On BC, a code object has a bit set if it's loaded at boot
time.
This change makes stack traces look more like Racket BC traces before
the macro expander was implemented in Racket. The frames for built-in
functions have been useful for implementing the expander and Racket
CS, but probably they're just noise for most users most of the time.
Set the `PLT_SHOW_BUILTIN_CONTEXT` environment variable to preserve
all available frames in the stack trace.
Some text-editing tools on Windows include a BOM character (encoded)
at the start of a file that is intended as UTF-8. The general
recommendation for UTF-8 is to *not* include a BOM --- but, well,
Windows. When a BOM is there, meanwhile, the recommendation is to
preserve it in the stream, so always discarding an initial BOM at the
file-port level is not a good idea. A new file mode would make sense,
but distinctions like 'text and 'binary mode have turned out to be
best avoided.
Although I'm not sure it's really a good idea, treating a BOM
character as whitespace in the reader (at least in comment positions)
is an easy way around the problem for text files that are intended as
programs.
Closes#1114
The `current-locale` parameer value is transferred to the C lbrary via
`setlocale` on demand by Racket functions that depend on the locale,
but some libraries (like libedit) can also depend on the locale
setting. By default, the C library starts with the "C" locale, so it's
worthwhile to install the default locale on startup, even if that's
not a complete solution to support changes to `current-locale`.
This fixes at least one "potential" bug in the file
`collects/pkg/private/create.rkt`, where `else` in the
`cond` is bound to `else` from `match` instead of `racket/base`.
(though it turns out that the format will always be
truthy, making the program happen to be correct.)
Without this change, the below build fails for me.
This reflects the GRACKETLDFLAGS in the makefile one folder above. However I have no idea if this has wider implications...
```
mkdir build
cd build
..\configure --enable-bcdefault
make
```
Change the ".zo" format to convert a linklet bundle hash table to a
list, which avoids probblems with stencil-vector encodings and cross
cimpilation between 32-bit and 64-bit platforms. Avoiding records,
stencil values, and hash code should make the fasled form simpler.
Refine the approach in 91abd020d1 so that it's only used when needed
to work for the combination of custodian management and unsafe
finalization.
Also, improve the documentation to clarify the constraints on
`register-finalize` due to its implementation in CS by ordered
finalization. This constraint is also reflected in a new `#:ordered?`
argument to `register-custodian-shutdown`. Any existing code that uses
`register-custodian-shutdown` plus `register-finalizer` directly
instead of `register-finalizer-and-custodian-shutdown` would need to
be updated for Racket CS, but code like that should be rare to
nonexistent.
Closes#3352
Reverts the repair attempt in 91abd020d1. The problem with switching
to a "late" reference is that it's based on ordered finalization in
Racket CS, which doesn't work on values that can refer back to
themselves.
Propagate `LIBS` to rktio's configure, and also move some flags in
`LIBS` that should be in `LDFLAGS`. The immediate result is to repair
the detection of iconv for rktio on FreeBSD.
Closes#3353
Change the regular weak reference in a custodian, which allows
`will-executor`-based finalization, to a "late" weakk reference, which
allows both `will-executor` and `register-finalizer` finalization.
Closes#3352
- the collector now promotes objects one generation higher at a time
by default. previously, it promoted every live oldspace object to
the selected target generation, which could result in objects
prematurely skipping one or more generations and thus being
retained longer than their ages justify. the biggest cost in
terms of code complexity and performance is the recording of
pointers from older newspace objects to younger newspace objects
that could not previously occur.
gc.c, alloc.c, externs.h
- the collect procedure now takes an additional optional minimum
target generation argument to allow the new default behavior to
be overridden.
7.ss, primdata.ss,
gcwrapper.c,
7.ms, root-experr*,
smgmt.stex, release_notes.stex
- added cn flag to control collect-notify
mats/Mf-base
- resweep_weak_pairs now sets sweep_loc to orig_next_loc rather than
first_loc since the latter could result in unnecessary sweeping of
existing target-generation weak pairs.
gc.c
- added set of S_child_processes[newg] to S_child_processes[oldg]
in S_do_gc code handling decreases in the maximum generation.
gcwrapper.c
- a specialized variant of the collector is used in the common case
where the max copied generation is 0, the min and max target
generations are 1, and there are no locked generation 0 objects
is now used. with the default collection parameters and no locking
of generation 0 objects, these collections account for 3/4 of all
collections.
gc.c, gc-011.c (new), gcwrapper.c, externs.h, c/Mf-base
- maybe-fire-collector no longer tries to be so precise and instead
just counts the number of generation-bytes allocated since the
last gc. suprisingly, rebuilding the s directory requires about
the same number of collections with this coarser (and less
expensive) measurement. this change also fixes a problem with
too-frequent collections when the maximum-generation is set to
zero. to make the determination even less expensive, a running
total of bytes in each generation is now maintained in a new
bytes_of_generation vector, and maybe-fire-collector is no longer
called when the collector is running.
alloc.c, gc.c, gcwrapper.c, globals.h
- copy now copies two pairs at once only if they are in the same
segment, which saves a few memonry references and tests and turns
out not to reduce the number of opportunities significantly in
tested programs.
gc.c
- occupied_segments, first_loc, base_loc, next_loc, bytes_left,
bytes_of_space, sweep_loc, and orig_next_loc are now indexed
by [g][s] rather than [s][g] to improve locality in the default
(and common) case where there are only a handful of active
generations.
globals.h, types.h, segment.c, gc.c, gcwrapper.c, prim5.c
- now maintaining 16-byte architectural stack alignment (if the
incoming stack is so aligned) on all x86 platforms except
i3nt/ti3nt. more recent versions of gcc sometimes generate sse
instructions that require 16-byte stack alignment.
x86.ss
[Merge for Racket includes additional changes to combine with in-place
marking - mflatt]
Incorrect initialization of the copy's iteration state for a hash
table meant that table sizes larger than 32 looked like tables of only
32 elements when iterating (including when printing).
Closes#3344
An optimization pass used mostly for inlining did not reqcognize
`quote`, and it could replace a quoted name with a constant-propagated
value.
Closes#3339
In initial-state, the call to dir-list will sort the results of directory-list. However, in initial-state, there is a different path that will call directory-list without sorting. This change makes these paths consistent.
The `flsingle` function takes a flonum and discards precision that
wouldn't fit in a single-flonum. If single-precision arithmetic is
somehow useful, then combine flonum operations with `flsingle` to
discard precision on the result; even on Racket BC, that's likely to
perform better than using generic arithmetic on single flonums.
The top-level makefile now builds Racket CS as `racket` by default.
Use `racket bc` to build Racket BC as `racket`. Use `make both` to
build both CS and BC (the latter with the `bc` suffix) overlayed in a
single build. By using `make both` insted of `make cs` plus `make bc`,
you can avoid redundant package downloads and documentation rendering.
To build Racket BC as `racket`, use `racket bc RACKETBC_SUFFIX=`, but
you must consistently use `RACKETBC_SUFFIX=` with `make` every time.
Adjust `configure` and makefiles so that `configure` can be run in an
directory, not just the source directory. Initial boot files are found
at either the current directory or the source directory.
Also, add support for `-B` and `--Boot` flags to accept a path that is
resolved relative to the current directory, instead of relative to the
installation.
The `--enable-cs` and `--enable-csdefault` flags now enabled just the
CS build, as long as pb boot files are present or
`--enable-racket=...` is supplied.
Currently, `make` is the same as `make bc`, but the idea is that
`make` can become the same as `make cs` when some other pieces are in
place.
The source of the top-level makefile is now ".makefile", and it's
turned into "Makefile" using "racket/src/makemake.rkt". The
transformation makes non-GNU `make` variants and `nmake` act like GNU
make to propagate variables, which makes abstraction through targets
plus variables (admitedly an abuse of `make`) more reliable and
consistent.
Why abuse `make` this way? Because `make` variants and `nmake` are
similar enough that to constitute a portable scripting language, and
one that conveniently provides a large number of entry points.
Also, schemified "thread", "io", "regexp", "schemify", and "expander"
layers are checked in. Overall, building Racket CS no longer requires
first building Racket BC.
The imported sources are from the `racket/ChezScheme` repository,
which branched from `cisco/ChezScheme`. The commit history is
preserved, but all of the commit IDs have shifted, because bootfiles
have been pruned from the history.
The "cs-bootstrap" package is now in the `racket/ChezScheme` repo,
because it needs to track the Chez Scheme implementation instead of
the Racket implementation.
Commit 023681947c fixed a problem cooperating with xform, but the
repair assumes xform. The "embed-in-c.rkt" test, for example, uses 3m
without xform (and instead uses macros like `MZ_GC_DECL_REG`.
It's not clear that adding to "HISTORY.txt" is all that useful. But as
long as we do, at least include milestones that were signficant enough
to mention in release announcements.
CS didn't always return a complete path when simplifying in
use-filesystem mode. On Windows, CS and BC were inconsistent with each
other and the Unix behavior.
A `\\?\RED\` path is Racket- and Windows-specific, and it's an extreme
corner case: a drive-relative absolute path that include elements that
must eb specially esacped. BC's `build-path` incorrectly allowed
`\\?\RED\` to extend an absolute path. CS's `build-path` incorrctly
allowed various absolute-path extensions, including `\\?\RED\` paths.
The documentation was slightly off.
Since file links and directory links on Windows are disjoint, and the
difference is relevant for operations such as deleting a file,
`link-exists?` is not enough information. Add `file-or-directory-type`
to provide more information and also avoid separate calls to
`file-exists?`, `directory-exists?`, etc.
The `delete-directory/files` function now uses `file-or-directory-type`
so that it will work right with Windows directory links.
Relevant to #3288
Procedures in compiled code could not previously have source-location
paths that are managed through the write-relative and load-directory
configuration. Instead, paths were always converted to strings that
start "..." --- and those strings were sometimes incorrectly converted
back to paths in context information extracted from a continuation
mark set.
This commit takes advantage of changes to Chez Scheme `fasl-write` and
`fasl-read` (and related for compiling code) to lift paths out where
linklet-level marshaling can take care of them.
Sync with changes from cisco/ChezScheme. The specific code fragments
that are compressed and the chunks that are used for compression
remain essentially the same as before for Racket CS, but a different
organization at the Chez Scheme level takes over some of the work that
was on the Racket CS linklet layer, and load times may improve
slightly.
In CS, if you interrupt an especially tight non-tail recursion, such
as
(let loop ()
(cons 1 (loop)))
then the "context" view of the continuation (as recorded in a
continuation mark set) can take space that is a multiple of the size
of the continuation itself. That's a particular problem if the
too-deep recursion triggers the memory limit in DrRacket, because
DrRacket will then need a multiple of its current heap space to report
"out of memory".
(Note: Just keeping the continuation itself is not a good option,
because that may retain other data referenced by the continuation.)
This commit reduces the heap space used to gather a continuation
context, relying in part on new Chez Scheme support, but mostly it
limits the context length to roughly the same maximum as in BC. The BC
limit is an implementation artifact, but it turns out to have good
properties; informaiton on more than 64k continuation frames is rarely
useful. The limit could be a parameter, but a large built-in limit
seems likely good enough.
(Another note: Adding a limit argument to
`continuation-mark-set->context` doesn't help enough, because it's too
late by that point; too much memory has been used to repersent the
information that's in the mark set.)
The commit also tightens tracking of continuations for memory
accounting, reducing the chance that a thread's large continuation
will be charged to the wrong custodian.
- A part of contract-out's code generation for struct assumes that
there's no parent struct and uses the provided struct name for
everything. This causes duplicate definitions when there are duplicate
field names where one is in a child struct and another is
in a parent struct. This PR fixes the problem.
- Disallow multiple #:omit-constructor
- Deprecate super-id. This information is unnecessary since we can
extract it from static struct information already. Attempting to
check that super-id is well-formed is error-prone due to how
the super struct type could be contracted which shields us from
detecting that they are indeed the super type.
- Utilize static struct field name information, and provide
the information when exporting a struct.
This PR is largely based on #732.
Fixes: #3266, #3269, #3271, and #3272
This commit adds `prop:struct-field-info` which is implemented to provide
static information about field names. The property is attached to all
struct types generated by `define-struct`.
The commit also modifies kernstructs to have the property.
Finally, the commit switches `struct-copy` to use the static field name
information when it's available.
It remains to change `contract-out` and `match`'s `struct*` to
recognize/attach this new property, but this could be done
separately in the future.
This PR is largely based on racket/racket#732, though the approach is slightly
different.
Although building with the wrong offsets is good enough to compile
the compiler to compile itself correctly, a broken intermediate
compilation can create confusion.
As the documentation says, `datum->syntax` should use
`datum-intern-literal`. That helps avoid syntax-object mutability, and
it increases sharing in compiled forms. The use of
`datum-intern-literal` got lost when the expander was rewritten in
Racket.
Relevant to #3245
The default package scope is determined by consulting the user-scope
configuration, then falling back to the installation-scope
configuration, then defaulting to user.
So, if you have a user-scope configuration of `default-scope`, it
doesn't matter what the configuration says in other scopes, which
means that the output of `raco pkg config` can be confusing. Extra
output in this commit is intended to make it less confusing.
Probably the original mistake here was allowing `default-scope` at a
scope-specific layer, instead of having it just as an installation
configuration (like `name`).
* Document liberal-defines? argument of generate-expand-context
* Add make-expression-transformer
* Ensure `block` expands in an expression context
Also refactors block to depend on racket/base and syntax/ modules rather
than private pre-base dependencies so it can use
`make-expression-transformer` and `generate-expand-context`.
* Ensure `local` expands in an expression context
* Add macro.rktl tests for block and local
An optimization relatively late in the BC bytecode compiler pipeline
was wrong for `begin0`. The transformation and bug must be a very old,
since it's intended to help the bytecode interpreter.
Thanks to Sage for reporting and Alexis for initial debugging.
When linking with libracket.a or libracket3m.a, librktio.a is needed.
(The instructions in "Inside" have apparently been wrong since rktio
was split out.)
The main (slightly) effective change here is to avoid disturbing loop
patterns within the Rumble layer's implementation.
Most of the commit is a commented out, updated version of the Scheme
implementation of MRG32k3a `random`. With the latest improvements for
unboxed floating-point arithmetic, performance is relatively good, but
it doesn't catch up to the C compiler's output. On an x86_64 MacBook
(i7 4870HQ) using LLVM or a Raspberry Pi 3 using GCC, it's about 50%
slower compared to C (in contrast to 300% slower before unboxing).
It's almost the same speed on a older x86_64 Linux machine (i7 2600)
using GCC. Where the C compiler wins, maybe it's due to the use of
SIMD instructions in the C output for x86_64 and Arm32. Switching to
the Scheme implementation of `random` would probably be fine, but
aisde from the satisfaction of being in Scheme, there's no reason to
pay the sometimes 50% penalty for now.
Caching compiled JIT fragments in a SQLite database did not turn out
to be a viable path, so remove partial support for it. JIT mode in
general is rarely a good option, but it's at least completely worked
out, so left in for now.
Update the Guide's performance section with current information for
Racket CS, and also document the Racket CS compilation mode and
inspection environment variables. Make a couple of environment
variables work more consistently: PLTDISABLEGC for CS and PLT_ZO_PATH
for BC.
When the runtime thread `touch`es a future that is blocked on an
atomic action (just as JIT compilation), the runtime thread would
eagerly run the action, but still leave the future on the
atomic-action queue. Atomic actions tend to be ok to run a second time
(including JIT compilation), so a problem may not show up immediately,
but a semaphore can get out of sync and cause problems later.
Change `fl->fx` to truncate as it converts, which is typically done
anyway by a machine instruction to convert from floating-point to
integer values. This makes `fl->fx` different from `inexact->exact`
or `fl->exact-integer`, but it brings BC and CS in line.
Follows Chez Scheme and Guile. Turns `(exp 10000.+0.0i)` into
`+inf.0+0.0i` instead of `+inf.0+nan.0i`, which is analagous to
the behavior for exact 0 in the complex part.
Fixes#3194.
If the callback takes too long to run, then a second copy was
scheduled --- which likely schedules a third copy, and so on. This
problem could lead DrRacket to get stuck in a GC cycle, for example.
In Schemify's lifting pass, avoid disturbing loop patterns that Chez
Scheme will recognize. Keeping the loops intact is helpful for
floating-point unboxing, where (currently) unbxoing is supported
across loop iterations but not across function boundaries in general.
When a `module` or `let-syntax` form is `expand`ed (i.e., when a
syntax object must be generated), then code is expanded to a syntax
object and then parsed again for compilation. In that second parse,
take advantage of the fact that the expression is already expanded,
which means that no new scopes or bindings need to be created.
Related to #3165
Two "self" module path indexes are treated the same on load when they
have the same resolved module name. Fix the serialization intern table
to take that into account and avoid some GC-based non-deterministism.
Related to #3165
On a second look, 6d06086dad unnecessarily duplicates work already don
by `copy-file` in most cases. For the case where `copy-file` was not
used, there was code to explicitly set an executable bit, but the new
approach is better and should be used just there.
Changes `_ptr`, `_list`, and `_vector` syntax to be more permissive.
Matching by symbol rather than binding is usually the wrong choice,
but the position where `i`, `o`, or `io` is recognized in `_ptr` and
company is not an expression position. Also, `i`, `o`, and `io` are
not otherwise bound. So, it's difficult to justify the current
behavior. If `_ptr` and company were defined today, we'd use keywords
like `#:i` instead of symbols like `i`.
A potential drawback of this change is that a `i`, `o`, or `io` might
be written with the intent of referring to a binding, and this change
makes such a reference silently allwowed instead of an errror. That
seems relatively unlikely, however, while having to avoid `i`, `o`, or
`io` as a binding is occassionally a pain --- something I've run into
once or maybe twice.
Closes#887
The Windows scheduler tends to work on 16ms bounaries, which
is not good for sleeping (as part of an animation, say) for
times near or less than 16ms. Change rktio to request more
fine-grained scheduling temporarily when trying to sleep for
a short time.
Extracting the name from a procedure involves `string->symbol` and
possibly some string parsing. Map the code object to the result
symbol to speed up multiple requests for the same code object.
* Ensure that the GMP ARM extra file has the correct extension
We hardcoded in some places the extension of the GMP ARM extra file
with .o, however the Makefile rule to build the file will use .lo is
--enable-shared is enabled. This commit fixes the discrepancy by not
hardcoding the extension anywhere.
Fixes#3176
* Avoid gnu make extension
* Replace spaces to conform with other lines
* Ensure we only add extension if file is needed
* Move the LTO definition to the configure.ac
* Carry the LTO variable into the Makefile
* Fix variable reference
* Add variable definition to gc2 Makefile
* annotate assignment
Concurrent lazy scope propagation had a (very unlikely) race condition
updating a syntax object. Ryan pointed out this problem as part of the
discussion for #3162.
Switch to using a single field for a syntax object's content and
propagations, so a CAS can be used to update the field. As a pleasant
side effect, this change tends to make syntax objects more compact.
Fix `ptr-ref` to properly handle the offset in a pointer for
specialized references, like extracting ` _double`.
Thanks to Laurent Orseau for the bug report and Jens Axel Søgaard for
intial debugging.
When a future triggers a write barrier, a lock is needed to prevent a
race between the future thread and other threads in the same place (on
most platforms).
Thanks to Dominik Pantůček for tracking down this bug.
Check `file-exists?` before passing the file on to `file-size` and/or
`open-input-file`. The latter led to confusing error messages, e.g.:
```
(file->string "DNE")
;; file-size: cannot get size
```
affects: `file->value` `file->list` `file->string` `file->bytes`
`file->lines` `file->bytes-lines`
The call to `random-sample/replacement` gets called when `(not
replacement?)` holds, while `random-sample/out-replacement` gets called
otherwise.
This is probably because the algorithm for the without replacement case
actually replaces parts of the sample, but that is different from the
meaning of a sample with and without replacement, which means with
possible duplicates or without.
This PR fixes four bugs:
1. Accessors are required at use-site in order to use `struct-copy`.
This PR removes that requirement since the information is already available in
struct-info. The following program used to fail prior the PR but will now pass.
```
(module a racket
(provide a)
(struct a (b)))
(require 'a)
(struct-copy a (a 1) [b 2])
```
2. `struct-copy` fails if the structure type transformer binding is renamed
(#1399). The following program used to fail prior the PR but will now pass.
```
(module struct racket/base
(provide (struct-out point))
(struct point (x y) #:transparent))
(require (rename-in 'struct [point point2d]))
(struct-copy point2d (point2d 1 2) [x 3])
```
3. With supertype, it's possible to construct colliding accessors,
causing `struct-copy` to update an incorrect field. The following program
produced incorrect outputs prior this PR but will now be correct.
```
(module a racket
(provide a)
(struct a (b-c) #:transparent))
(require 'a)
(struct a-b a (c) #:transparent)
(struct-copy a-b (a-b 1 2) [b-c #:parent a 10])
;; before the PR: (a-b 1 10), after the PR: (a-b 10 2)
(struct-copy a-b (a-b 1 2) [c 10])
;; before the PR: (a-b 1 10), after the PR: (a-b 1 10)
```
4. Similar to 3., prior this commit, it's possible to refer to a bogus field
name when supertype is present. The following program doesn't result in
a syntax error which is wrong. This commit fixes that.
```
(module a racket/base
(provide (all-defined-out))
(struct a (b-c) #:transparent))
(require 'a)
(struct a-b a (d) #:transparent)
(struct-copy a-b (a-b 1 2) [c 10])
```
The key idea is that the actual struct name (if the struct is created via
`struct` or `define-struct`) can be extracted from the name of struct predicate.
The actual field names then can be precisely extracted from accessors.
Note that struct-infos that are created manually by `make-struct-info`
didn't work with `struct-copy`. This PR didn't attempt to fix that because
it requires a significant change that would not be backward compatible with
the current struct info.
Instead of logging an error when the `openssl` module is loaded, defer
a complaint until procedures that would depend on the configuration is
called. Otherwise, errors can get printed in programs that depend on
the `openssl` library but do not always need OpenSSL support at run
time.
This is a backward-incompatible changed, but no packages currently
registered at pkgs.racket-lang.org refer to `ssl-dh4096-param-path`.
Providing `ssl-dh4096-param-bytes`, instead, avoids carrying along an
extra file with any stand-alone executable that depends on `openssl`.
Don't try to park values when allocating a weak box or pair in a
future thread, since that creates a race on the parking spaces. A
future thread can't run a GC, so it's doesn't need to park.
Touching a future in a future allocates a weak box, so this bug could
have been responsible for many crahses.
Related to #3145
Update GMP license statment following 392dc33ceb which comes from
a recent GMP version.
This does not affect the overall license situation of Racket BC,
which includes other LGPL v3 code, or of Racket CS, which does not
use GMP.
The `call-with-deep-time-limit` function in `racket/sandbox` expects a
subprocess to be removed from its custodian when the subprocess is
done. CS wasn't doing that at all, leaving custodian removal to a
finalizer. BC was doing delaying a remove until `subprocess-status` is
used (which happened to work for existing uses of
`call-with-deep-time-limit`, apparently.)
Relevant to #3140
When incremental mode is enabled, adjust garbage collection to avoid
promoting objects from the next-to-oldest generation into the oldest
generation. This change produces a good approximation to incremental
collection for game-like programs (although probably not server-like
programs with large, temporary jobs).
Fix error checking and reporting for position-based struct accessors
and mutators. Also, fix mutability recording for prefab structure
types that have auto fields.
Related to racket/typed-racket#902
Implement 'atomic-interior allocation and immobile cells using
`make-immobile-bytevector` and `make-immobile-vector`, which avoids
having to unlock through a finalizer.
Also, the Chez Scheme GC can now mostly mark a major generation,
instead of copying it, which can significantly reduce memory
use during a GC for an old, large heap (such as DrRacket's).
Show peak administrative as a parenthesized delta on peak space.
For BC, this extra delta is small, because BC compacts (instead of
copying) old-generation objects. For CS, the extra delta can large ---
typically an extra 50%, but potentially another 100% --- because a
full collection copies all old-generation objects.
Also, for BC, fix cumulative-allocation reporting to include child
places.
Extract invert_limb code for ARM from GMP 6.2.
In order to check for thumb mode availability use defined macro `__thumb__`, which in turns requires us to process the source file with `gcc` instead of `as` in order to access the preprocessor - tested with `clang` as well.
Fixes#3050
The Chez Scheme change avoids a leak while collecting in counting
mode, which is used by Racket's memory-accounting mode.
Also, add a small repair for 4256214981.
GitHub switched URLs for tarballs, redirecting to the new one with
a 302. However, old versions of Racket don't follow redirects in
`raco pkg install`, so they broke (before 6.3). Using the new URL
should work for everyone.
Reported by @greghendershott.
Also `--include-deps`, which support the creation of a catalog archive
that is restricted to a specific set of packages. Also
`--fast-file-copy`, which is usefl for speeding up a pipeline of
archiving (helpful to pkg-build).
Chez Scheme now supports a single-pass combination of `collect` and
`compute-size-increments`, which makes a GC with accounting about
twice as fast. Meanwhile, other GC improvements reduce non-accounting
full-collection times by 10-20%.
Much of the GC implementation is now generated from a "Parenthe-C"
description, so update the bootstrap process for that step.
This fixes a major issue in arm 32bits, detected by ubsan, where
conversion of type to milliseconds results in overflow.
```
rktio_time.c:92:21: runtime error: signed integer overflow: 1584975753 * 1000 cannot be represented in type 'long int'
```
GCC 9.3 things that `h2` might reach line 385 uninitialized.
```
./hash.c:385:14: warning: 'h2' may be used uninitialized in this function [-Wmaybe-uninitialized]
385 | h = (h + h2) & mask;
| ~~~^~~~~
```
A Chez Scheme garbage collection involves a rendezvous among threads
that are used to implement places and futures (and potentially other
things that create Chez-level threads). The thread that is used to
drive the garbage collection was not formerly specified, and callbacks
like the GC icon in DrRacket can only be run in the initial thread.
When a major collection was run in a non-initial thread, the callback
was simply skipped.
The Racket branch of Chez Scheme now drives a collection in the
initial thread whenever that thread is active. In Racket CS, the
initial thread can be inactive if it's waiting for external events,
and a place can meanwhile trigger a GC. Now, when the Racket CS GC
callback is called for a major collection in any thread other than the
initial one, it defers the major collection to an asynchornous
callback in the main thread (and meanwhile performs a minor
collection).
So, a major collection might now be delayed by just a little while if
the main thread is inactive, such as when it's waiting for external
events, but callbacks like the GC icon in DrRacket will be reliably
invoked for major collections.
It's still unclear what the specification of syntax->string regarding whitespace
before closing parens should be. The implementation also has not dealt with
the issue. This PR therefore removes whitespace before closing parens from
the tests.
Once the discussion at
https://github.com/racket/racket/issues/3071#issuecomment-601984438
has reached a conclusion and implemented, we can add these tests back.
The `call-in-continuation` function generalizes applying a
continuation to values by accepting a thunk that is called in the
restored continuation. In other words, insteda of having to use the
pattern
((call/cc (lambda (k)
.... (set! saved-k k) ...
(lambda ()
original-result))))
...
(saved-k (lambda () new-result))
The extra call and thunk on the capture side can be omitted:
(call/cc (lambda (k)
.... (set! saved-k k) ...
original-result))
...
(call-in-continuation saved-k (lambda () new-result))
At the Chez Scheme level, a `call-in-continuation` in tail position
within a function can avoid forming a closure for its second argument.
The `call-in-continuation` function at the Racket CS level doesn't yet
provide that benefit.
The `call-in-continuation` operation is called `continuation-slice` in
Feeley's "A Better API for First-Class Continuations".
Expansion of a procedure with keywords is quadratic due to generating
a nested sequence of `let`s, but speed it up by roughly a constant
factor by using a dintinct symbol for each nested layer.
Related to #3070
Using `call-in-continuation` to apply a thunk within a continuation
slightly simplifies and speeds up parts of the implementation of
delimited continuations.
The expansion of `struct` created far too much code to parameterize
`struct-field-index`, making expansion of a `struct` form with just
100 or 200 fields take a noticeably long time to expand.
PR #2678 unintentionally removed this attribute, but it was used
at least by "collections-lib" and "static-rename-lib".
cc @sorawee @lexi-lambda @jackfirth @rmculpepper
GitHub's CDN seems to have recently started returning the `Location` header for a redirect with a lowercase `l`, which breaks the redirect logic. The HTTP spec says that header names are case-insensitive, so we need to look for either version.
Make `register-process-global` check for byte strings, and avoid
retaining the byte string that it's given (in case that changes, for
example).
Closes#3053
Building glib-specific support into the main Racket executable is
unsatisfying, but it's consistent with Racket BC, and the alternative
is especially tedious to deal with places and namespaces and
allocation.
Combined with a schemify improvement that inlines imported predicates
to expose the record-type test to Chez Scheme, this chage enables
cptypes to prune useless inlined `wrapped-object` selector branches.
That improvement, in turn, reduces code size and redundant checks on
objects that have contracts.
When a `lambda` form is marked as a method (for arity error reporting)
through a property, translate that to a static flag on the procedure,
instead of a call to `procedure->method`.
The only way we have to attach static information is through the
procedure name, so the encoding already in place for "no name" and
"path-based name" is extended to support a method flag.
Since procedures have names that are attached more directly by the
schemify pass, remove simple `let-values` forms wrapping procedures.
This shortcut improves the result of the lifting pass in some cases.
Record types with unnamed fields can be significantly more compact,
excdeption in combination with the constrain the the fields all
contain Scheme objects. Saves 2% for DrRacket's initial footprint.
We had to give back some space savings to avoid potential problems
with allocating in the event callback and running out of stack
space[?] when passing an argument. We get to keep most of the
improvement though.
Fix a regression in 712494312a, and change other other two printers to
be more consistent for
#lang racket
(struct s () #:transparent)
(define a (s))
(pretty-print (list (cons a 0) (cons a 0)))
A regexp can match while having sub-patterns that are not used in
the match. In regexp-replace, the "insert" argument can refer to
these unused sub-matches. If a function is given for the "insert"
argument, it will receive these unused submatches as #f values.
If a string or byte-string is given, then any reference to an
unused sub-match (by means of a back-reference) is treated as the
empty string or byte-string.
Fixes#3032.
When logging GC debugging, a pecentage after the time for a GC reports
what fraction was extra steps after GCing proper, especially the extra
step of memory acounting when that is enabled.
Also, avoid Chez Scheme gensyms even more. Otherwise, using low-level
facilities like `dump-memory-stats` can force the unique name of a
gensym used for a structure type, which causes it to be permanent,
which could be exactly what you don't want when debugging a
memory-rentention problem.
In `compiler/private/mach-o` --- which is reachable via `racket` due
to being a dependency of `setup/dirs` --- delay the call to
`cross-system-library-subpath` until needed.
Fixes optimization for an expression like
(define (f x)
(lambda (y)
(letrec ([recursion (f x)])
(+ x y))))
by adjusting the inlining hueristic to support less inlining on a
second pass of `letrec` right-hand sides.
Closes#3027
Provide `vm-primitive` and `vm-eval` to regularize access to VM-level
primitives. Document some of the issues for interoperating across the
Racket and (Chez Scheme) VM layers.
The library could have been implemented with just `compile-linklet`
and `instantiate-linklet`, but using an underlying `primitive-lookup`
function is a little nicer.
Produce 0, -1, or out-of-memory for bignum shifts. For large fixnum
shifts, check memory limits.
The repairs are mostly for Racket CS, but traditional Racket
incorrectly reported out-of-memory for 0 shifted by a positive bignum.
A more compact representation for return points tends to reduce code
by about 10%. For DrRacket, that translates to a 5% decrease in
overall footprint.
Explain how to deal with the "racket-lib" that is normally included
with a source distribution, but that doesn't have dependencies
specific to Windows or Mac OS.
Since schemify adds its own checking for procedureness in an
application, always compile the application as unsafe at the Chez
Scheme level. This simple change saves about 5% in code size for
DrRacket, which is a 1-2% footprint saving overall.
An improvement to Chez Scheme allows more function from the Rumble and
other built-in layers to be inlined into compiled Racket code, and a
new `$app/no-inline` primitive enables improved control over how slow
paths are integrated.
The pretty printer and built-in printer for traditional Racket did not
consistently provide the current quoting mode while checking for
unquoting and cycles. All printers, including the Racket CS printer,
are improved for a structure type that has
`prop:custom-print-quotable` as 'always, in which case we know that
unquoting- and cycle-checking time that the components will be in
quoted mode.
The pretty printer also made three passes through a value to check for
cycles, compute cycles, and compute unquotes, and those are now fused
into a single pass like the Racket CS printer. The built-in printer
for traditional Racket still makes up to two passes, but it now
behaves more like other printers by recurring immediately on nested
calls via `prop:custom-write` instead of accumulating them for after
the `prop:custom-write` callback returns.
The documentation clarifies that synthesizing new values during
printing can interefere with cycle checking and unquoting, but the
printers now react to that behavior more consistently.
With recent improvements, the run-time performance of vector-stencil
HAMTs for immutable hash tables seems close enough (on
microbenchmarks) to the Patricia-trie implementation to be worthwhile,
since they use less memory. Performance remains better in most cases
than the traditional Racket implementation.
The table at the end of this message summarizes relative performance
on microbenchmarks. Overall, though, immutable hash-table operations
are already so fast that these difference very rarely translate to
measurable differences in overall run times --- not even for the macro
expander, which relies heavily on immutable hash tables to represent
scope sets.
Stencil-vector HAMTs tend to take about 1/3 the space of Patricia
tries, and those space savings can turn into run-time improvements in
applications by reducing GC time. I've observed a 10% reduction in
compile time for some programs. When building a full Racket
distribution, run time shrinks by about 2 minutes out of 80 minutes,
probbaly because just average memory use goes down by 10%. DrRacket's
initial memory footprint goes down by about 37M out of 657M (a 5%
savings).
Mincrobenchmark relative performance, normalized to previous Racket CS
implementation (measured on 2018 MacBook Pro, 2.7 GHz Core i7; Chez
Scheme can substitute POPCNT instructions at link time):
patricia = previous Racket CS implementation as a Patricia Trie
stencil = new Racket CS implementation as a stencil-vector HAMT
racket = traditional Racket implementation
patricia stencil racket
set-in-empty:eq#t: ==| ==| ==|=
set-many:eq#t: ==| ==|== ==|========
set-many-in-order:eq#t: ==| ==| ==|====
set-same:eq#t: ==| == ==|=
set-in-empty:eq: ==| == ==|=
set-many:eq: ==| ==|== ==|========
set-many-in-order:eq: ==| ==|= ==|=====
set-same:eq: ==| == ==|=
set-in-empty:eqv: ==| ==| ==|==
set-many:eqv: ==| ==|== ==|=========
set-many-in-order:eqv: ==| ==|= ==|=====
set-same:eqv: ==| ==| ==|=
set-in-empty:equal: ==| ==|== ==|===
set-many:equal: ==| ==|== ==|=====
set-many-in-order:equal: ==| ==|= ==|===
set-same:equal: ==| ==|= ==|===
ref:eq#t: ==| ==| ==|=
ref-fail:eq#t: ==| ==| ==
ref:eq: ==| ==| ==|=
ref-fail:eq: ==| ==| ==
ref:eqv: ==| ==| ==|====
ref-fail:eqv: ==| ==| ==|
ref:equal: ==| ==| ==|===
ref-large:equal: ==| ==| ==
ref-fail:equal: ==| ==| ==|===
ref-large-fail:equal: ==| ==| ==
removes:eq#t: ==| ==|=== ==|===========
add+remove:eq#t: ==| ==|= ==|=======
removes:eq: ==| ==|==== ==|============
add+remove:eq: ==| ==|= ==|=======
removes:eqv: ==| ==|=== ==|=============
add+remove:eqv: ==| ==| ==|========
removes:equal: ==| ==|== ==|=======
add+remove:equal: ==| ==|= ==|======
iterate-keys:eq: ==| ==| ==|=
iterate-vals:eq#t: ==| ==|= ==|=
iterate-vals:eq: ==| ==|= ==|=
iterate-unsafe-keys:eq: ==| ==| ==|=======
iterate-unsafe-vals:eq#t: ==| ==| ==|
iterate-unsafe-vals:eq: ==| ==|= ==|
for-each:eq: ==| ==| ==|==========
subset-lil-shared:eq: ==| ==| ==|=
subset-lil-unshared:eq: ==| ==| ==|==
subset-lil-not:eq: ==| == ==
subset-med+lil-shared:eq: ==| ==|==== ==|=
subset-med+med-shared:eq: ==| ==|= ==|=
subset-big-same:eq: ==| ==| ==|===============
subset-big+lil-shared:eq: ==| ==|=== ==|====
subset-big+med-shared:eq: ==| ==|== ==|===
subset-big-unshared:eq: ==| ==| ==|==
This commit adds an (unused) implementation of immutable hash tables
for Racket CS that trades some run-time performance for an especially
compact representation --- similar to the traditional Racket
implementation of immutable hash tables. It uses a new "stencil
vector" datatype at the Chez Scheme level, which overlays the bitmap
needed for a HAMT node with the Chez-object type tag (and also
provides an update operation that avoids unnecessary memory work).
Compared to the current Racket CS implementation, the stencil-vector
HAMT implementation of an immutable hash table takes only about 1/3
the space on avergae, which translates to a overall 5% savings in
DrRacket's initial heap. It also makes a full Racket build slightly
faster by reducing avergage memory use by 5-10%.
But the run-time performance difference is significant, especially for
the `hash-keys-subset?` operation (at least in microbenchmarks), and
also for addition and iteration. Maybe there's an overall better point
that reduces memory use of the current Patricia trie implementation
without sacrificing as much performance.
Besides the benchmarks and stencil-vector HAMT implementaiton, there
are small changes to the way hash tables cooperate with `equal?`,
which makes it a little easier to plug in different implementations.
Building with shared libraries is not currently supported, because the
Chez Scheme build is not set up to work in that mode, and because
"stand alone" executable handling at the Racket level does not support
Racket CS shared libraries.
Also, there's no benefit to shared libraries. Racket executables get
the benefit of sharing because they all run through the same
executable. Meanwhile, there's not (yet?) a supported C API to make
something like "libracketcs.so" useful.
Related to #2993
Adjust part of the internal scheduling protocol to make a retry
callback generated by another callback that sets up the retry. This
helps clarify the protocol and avoids allocating a closure that is
rarely used.
Make Racket CS consistent with traditional Racket in the way
`chaperone-evt` on a thread hides threadness, etc.
Hiding properties like threadness is not ideal and does not seem
entirely consistent with `chaperone-of`, but allowing things like
threads and semaphores to be chaperoned creates non-trivial expense
internally. It would have been better to have event constructors for
threads and such to (and then the consyructed events could be
chaperoned without imposing a cost on the original data structure).
Force inlining of value and key accessors. Keeping the `define` with a
loop body --- that is, not making the individual function a macro ---
allows the Rumble `define` to avoid a closure allocation for the loop.
Hash iteration can fail if a GC collects a key in between
`hash-iterate-next` and `hash-iterate-key` (and similar). Use the
optional extra argument internally to detect and handle that case.
THe internal `assoc?` predicate is relatively expensive, so remove
redundant uses. Also, uses a cache to make it a little cheaper for
muliple uses of dictionary functions on a moderately sized list.
We have also removed ubsan flags from CPPFLAGS - since these are
compile flags, not preprocessing flags.
To pass the flags to rktio, we split the use of CFLAGS and CPPFLAGS
in rktio and ensure that flags in rktio do not update flags if make
is called in sub-make form (in a standard racket build, the rktio make
is called as a sub-make from the main racket make).
Fixes#2984
* Avoid export on variables in Makefiles
Instead pass the variables explicitly to the make call.
* Fix variable syntax
* Double quote variables
* Put ubsan back in CPPFLAGS
* Remove conditional assignment
This is not necessary if passing variables straight into sub-make
Instead of compiling the right-hand side of a `let-syntax`, interpret
using the schemify layer's safe-for-space interpreter. Similarly,
interpret the right-hand side of `define-syntax` when it is evaluated
during the enclosing module's expansion (compiling it for use when the
enclosing module is instantiated for expanding other modules).
This is an old idea, and it's effective in some cases: `racketcs -cl
racket/base` or `racketcs -cl racket` takes 20% less time with this
change. Various obstacles kept us from trying this earlier; most
significantly, the gap to finish the safe-for-space interpreter was
small enough to make it work. It's not clear that the result is an
improvement for `raco setup`, though.
The interpreter is not fast (it's about 1/4 the speed of the
traditional Racket interpreter), so there's room for improvement,
but even a slow interpreter pays off for building.
When recompiling modules from machine-independent mode to
machine-specific mode, dependencies of the module to be recompiled
must be checked explicitly, but using `module-path-index-resolve` is
not a good strategy, because it doesn't load transitive dependencies.
That can create a failure later when a transitive dependency is needed
for compilation and it isn't loaded because the dependent module was
already loaded.
Relevant to #2928
