Chez Scheme now supports a single-pass combination of `collect` and
`compute-size-increments`, which makes a GC with accounting about
twice as fast. Meanwhile, other GC improvements reduce non-accounting
full-collection times by 10-20%.
Much of the GC implementation is now generated from a "Parenthe-C"
description, so update the bootstrap process for that step.
This fixes a major issue in arm 32bits, detected by ubsan, where
conversion of type to milliseconds results in overflow.
```
rktio_time.c:92:21: runtime error: signed integer overflow: 1584975753 * 1000 cannot be represented in type 'long int'
```
GCC 9.3 things that `h2` might reach line 385 uninitialized.
```
./hash.c:385:14: warning: 'h2' may be used uninitialized in this function [-Wmaybe-uninitialized]
385 | h = (h + h2) & mask;
| ~~~^~~~~
```
A Chez Scheme garbage collection involves a rendezvous among threads
that are used to implement places and futures (and potentially other
things that create Chez-level threads). The thread that is used to
drive the garbage collection was not formerly specified, and callbacks
like the GC icon in DrRacket can only be run in the initial thread.
When a major collection was run in a non-initial thread, the callback
was simply skipped.
The Racket branch of Chez Scheme now drives a collection in the
initial thread whenever that thread is active. In Racket CS, the
initial thread can be inactive if it's waiting for external events,
and a place can meanwhile trigger a GC. Now, when the Racket CS GC
callback is called for a major collection in any thread other than the
initial one, it defers the major collection to an asynchornous
callback in the main thread (and meanwhile performs a minor
collection).
So, a major collection might now be delayed by just a little while if
the main thread is inactive, such as when it's waiting for external
events, but callbacks like the GC icon in DrRacket will be reliably
invoked for major collections.
It's still unclear what the specification of syntax->string regarding whitespace
before closing parens should be. The implementation also has not dealt with
the issue. This PR therefore removes whitespace before closing parens from
the tests.
Once the discussion at
https://github.com/racket/racket/issues/3071#issuecomment-601984438
has reached a conclusion and implemented, we can add these tests back.
The `call-in-continuation` function generalizes applying a
continuation to values by accepting a thunk that is called in the
restored continuation. In other words, insteda of having to use the
pattern
((call/cc (lambda (k)
.... (set! saved-k k) ...
(lambda ()
original-result))))
...
(saved-k (lambda () new-result))
The extra call and thunk on the capture side can be omitted:
(call/cc (lambda (k)
.... (set! saved-k k) ...
original-result))
...
(call-in-continuation saved-k (lambda () new-result))
At the Chez Scheme level, a `call-in-continuation` in tail position
within a function can avoid forming a closure for its second argument.
The `call-in-continuation` function at the Racket CS level doesn't yet
provide that benefit.
The `call-in-continuation` operation is called `continuation-slice` in
Feeley's "A Better API for First-Class Continuations".
Expansion of a procedure with keywords is quadratic due to generating
a nested sequence of `let`s, but speed it up by roughly a constant
factor by using a dintinct symbol for each nested layer.
Related to #3070
Using `call-in-continuation` to apply a thunk within a continuation
slightly simplifies and speeds up parts of the implementation of
delimited continuations.
The expansion of `struct` created far too much code to parameterize
`struct-field-index`, making expansion of a `struct` form with just
100 or 200 fields take a noticeably long time to expand.
PR #2678 unintentionally removed this attribute, but it was used
at least by "collections-lib" and "static-rename-lib".
cc @sorawee @lexi-lambda @jackfirth @rmculpepper
GitHub's CDN seems to have recently started returning the `Location` header for a redirect with a lowercase `l`, which breaks the redirect logic. The HTTP spec says that header names are case-insensitive, so we need to look for either version.
Make `register-process-global` check for byte strings, and avoid
retaining the byte string that it's given (in case that changes, for
example).
Closes#3053
Building glib-specific support into the main Racket executable is
unsatisfying, but it's consistent with Racket BC, and the alternative
is especially tedious to deal with places and namespaces and
allocation.
Combined with a schemify improvement that inlines imported predicates
to expose the record-type test to Chez Scheme, this chage enables
cptypes to prune useless inlined `wrapped-object` selector branches.
That improvement, in turn, reduces code size and redundant checks on
objects that have contracts.
When a `lambda` form is marked as a method (for arity error reporting)
through a property, translate that to a static flag on the procedure,
instead of a call to `procedure->method`.
The only way we have to attach static information is through the
procedure name, so the encoding already in place for "no name" and
"path-based name" is extended to support a method flag.
Since procedures have names that are attached more directly by the
schemify pass, remove simple `let-values` forms wrapping procedures.
This shortcut improves the result of the lifting pass in some cases.
Record types with unnamed fields can be significantly more compact,
excdeption in combination with the constrain the the fields all
contain Scheme objects. Saves 2% for DrRacket's initial footprint.
We had to give back some space savings to avoid potential problems
with allocating in the event callback and running out of stack
space[?] when passing an argument. We get to keep most of the
improvement though.
Fix a regression in 712494312a, and change other other two printers to
be more consistent for
#lang racket
(struct s () #:transparent)
(define a (s))
(pretty-print (list (cons a 0) (cons a 0)))
A regexp can match while having sub-patterns that are not used in
the match. In regexp-replace, the "insert" argument can refer to
these unused sub-matches. If a function is given for the "insert"
argument, it will receive these unused submatches as #f values.
If a string or byte-string is given, then any reference to an
unused sub-match (by means of a back-reference) is treated as the
empty string or byte-string.
Fixes#3032.
When logging GC debugging, a pecentage after the time for a GC reports
what fraction was extra steps after GCing proper, especially the extra
step of memory acounting when that is enabled.
Also, avoid Chez Scheme gensyms even more. Otherwise, using low-level
facilities like `dump-memory-stats` can force the unique name of a
gensym used for a structure type, which causes it to be permanent,
which could be exactly what you don't want when debugging a
memory-rentention problem.
In `compiler/private/mach-o` --- which is reachable via `racket` due
to being a dependency of `setup/dirs` --- delay the call to
`cross-system-library-subpath` until needed.
Fixes optimization for an expression like
(define (f x)
(lambda (y)
(letrec ([recursion (f x)])
(+ x y))))
by adjusting the inlining hueristic to support less inlining on a
second pass of `letrec` right-hand sides.
Closes#3027
Provide `vm-primitive` and `vm-eval` to regularize access to VM-level
primitives. Document some of the issues for interoperating across the
Racket and (Chez Scheme) VM layers.
The library could have been implemented with just `compile-linklet`
and `instantiate-linklet`, but using an underlying `primitive-lookup`
function is a little nicer.
Produce 0, -1, or out-of-memory for bignum shifts. For large fixnum
shifts, check memory limits.
The repairs are mostly for Racket CS, but traditional Racket
incorrectly reported out-of-memory for 0 shifted by a positive bignum.
A more compact representation for return points tends to reduce code
by about 10%. For DrRacket, that translates to a 5% decrease in
overall footprint.
Explain how to deal with the "racket-lib" that is normally included
with a source distribution, but that doesn't have dependencies
specific to Windows or Mac OS.
Since schemify adds its own checking for procedureness in an
application, always compile the application as unsafe at the Chez
Scheme level. This simple change saves about 5% in code size for
DrRacket, which is a 1-2% footprint saving overall.
An improvement to Chez Scheme allows more function from the Rumble and
other built-in layers to be inlined into compiled Racket code, and a
new `$app/no-inline` primitive enables improved control over how slow
paths are integrated.
The pretty printer and built-in printer for traditional Racket did not
consistently provide the current quoting mode while checking for
unquoting and cycles. All printers, including the Racket CS printer,
are improved for a structure type that has
`prop:custom-print-quotable` as 'always, in which case we know that
unquoting- and cycle-checking time that the components will be in
quoted mode.
The pretty printer also made three passes through a value to check for
cycles, compute cycles, and compute unquotes, and those are now fused
into a single pass like the Racket CS printer. The built-in printer
for traditional Racket still makes up to two passes, but it now
behaves more like other printers by recurring immediately on nested
calls via `prop:custom-write` instead of accumulating them for after
the `prop:custom-write` callback returns.
The documentation clarifies that synthesizing new values during
printing can interefere with cycle checking and unquoting, but the
printers now react to that behavior more consistently.
With recent improvements, the run-time performance of vector-stencil
HAMTs for immutable hash tables seems close enough (on
microbenchmarks) to the Patricia-trie implementation to be worthwhile,
since they use less memory. Performance remains better in most cases
than the traditional Racket implementation.
The table at the end of this message summarizes relative performance
on microbenchmarks. Overall, though, immutable hash-table operations
are already so fast that these difference very rarely translate to
measurable differences in overall run times --- not even for the macro
expander, which relies heavily on immutable hash tables to represent
scope sets.
Stencil-vector HAMTs tend to take about 1/3 the space of Patricia
tries, and those space savings can turn into run-time improvements in
applications by reducing GC time. I've observed a 10% reduction in
compile time for some programs. When building a full Racket
distribution, run time shrinks by about 2 minutes out of 80 minutes,
probbaly because just average memory use goes down by 10%. DrRacket's
initial memory footprint goes down by about 37M out of 657M (a 5%
savings).
Mincrobenchmark relative performance, normalized to previous Racket CS
implementation (measured on 2018 MacBook Pro, 2.7 GHz Core i7; Chez
Scheme can substitute POPCNT instructions at link time):
patricia = previous Racket CS implementation as a Patricia Trie
stencil = new Racket CS implementation as a stencil-vector HAMT
racket = traditional Racket implementation
patricia stencil racket
set-in-empty:eq#t: ==| ==| ==|=
set-many:eq#t: ==| ==|== ==|========
set-many-in-order:eq#t: ==| ==| ==|====
set-same:eq#t: ==| == ==|=
set-in-empty:eq: ==| == ==|=
set-many:eq: ==| ==|== ==|========
set-many-in-order:eq: ==| ==|= ==|=====
set-same:eq: ==| == ==|=
set-in-empty:eqv: ==| ==| ==|==
set-many:eqv: ==| ==|== ==|=========
set-many-in-order:eqv: ==| ==|= ==|=====
set-same:eqv: ==| ==| ==|=
set-in-empty:equal: ==| ==|== ==|===
set-many:equal: ==| ==|== ==|=====
set-many-in-order:equal: ==| ==|= ==|===
set-same:equal: ==| ==|= ==|===
ref:eq#t: ==| ==| ==|=
ref-fail:eq#t: ==| ==| ==
ref:eq: ==| ==| ==|=
ref-fail:eq: ==| ==| ==
ref:eqv: ==| ==| ==|====
ref-fail:eqv: ==| ==| ==|
ref:equal: ==| ==| ==|===
ref-large:equal: ==| ==| ==
ref-fail:equal: ==| ==| ==|===
ref-large-fail:equal: ==| ==| ==
removes:eq#t: ==| ==|=== ==|===========
add+remove:eq#t: ==| ==|= ==|=======
removes:eq: ==| ==|==== ==|============
add+remove:eq: ==| ==|= ==|=======
removes:eqv: ==| ==|=== ==|=============
add+remove:eqv: ==| ==| ==|========
removes:equal: ==| ==|== ==|=======
add+remove:equal: ==| ==|= ==|======
iterate-keys:eq: ==| ==| ==|=
iterate-vals:eq#t: ==| ==|= ==|=
iterate-vals:eq: ==| ==|= ==|=
iterate-unsafe-keys:eq: ==| ==| ==|=======
iterate-unsafe-vals:eq#t: ==| ==| ==|
iterate-unsafe-vals:eq: ==| ==|= ==|
for-each:eq: ==| ==| ==|==========
subset-lil-shared:eq: ==| ==| ==|=
subset-lil-unshared:eq: ==| ==| ==|==
subset-lil-not:eq: ==| == ==
subset-med+lil-shared:eq: ==| ==|==== ==|=
subset-med+med-shared:eq: ==| ==|= ==|=
subset-big-same:eq: ==| ==| ==|===============
subset-big+lil-shared:eq: ==| ==|=== ==|====
subset-big+med-shared:eq: ==| ==|== ==|===
subset-big-unshared:eq: ==| ==| ==|==
This commit adds an (unused) implementation of immutable hash tables
for Racket CS that trades some run-time performance for an especially
compact representation --- similar to the traditional Racket
implementation of immutable hash tables. It uses a new "stencil
vector" datatype at the Chez Scheme level, which overlays the bitmap
needed for a HAMT node with the Chez-object type tag (and also
provides an update operation that avoids unnecessary memory work).
Compared to the current Racket CS implementation, the stencil-vector
HAMT implementation of an immutable hash table takes only about 1/3
the space on avergae, which translates to a overall 5% savings in
DrRacket's initial heap. It also makes a full Racket build slightly
faster by reducing avergage memory use by 5-10%.
But the run-time performance difference is significant, especially for
the `hash-keys-subset?` operation (at least in microbenchmarks), and
also for addition and iteration. Maybe there's an overall better point
that reduces memory use of the current Patricia trie implementation
without sacrificing as much performance.
Besides the benchmarks and stencil-vector HAMT implementaiton, there
are small changes to the way hash tables cooperate with `equal?`,
which makes it a little easier to plug in different implementations.
Building with shared libraries is not currently supported, because the
Chez Scheme build is not set up to work in that mode, and because
"stand alone" executable handling at the Racket level does not support
Racket CS shared libraries.
Also, there's no benefit to shared libraries. Racket executables get
the benefit of sharing because they all run through the same
executable. Meanwhile, there's not (yet?) a supported C API to make
something like "libracketcs.so" useful.
Related to #2993
Adjust part of the internal scheduling protocol to make a retry
callback generated by another callback that sets up the retry. This
helps clarify the protocol and avoids allocating a closure that is
rarely used.
Make Racket CS consistent with traditional Racket in the way
`chaperone-evt` on a thread hides threadness, etc.
Hiding properties like threadness is not ideal and does not seem
entirely consistent with `chaperone-of`, but allowing things like
threads and semaphores to be chaperoned creates non-trivial expense
internally. It would have been better to have event constructors for
threads and such to (and then the consyructed events could be
chaperoned without imposing a cost on the original data structure).
Force inlining of value and key accessors. Keeping the `define` with a
loop body --- that is, not making the individual function a macro ---
allows the Rumble `define` to avoid a closure allocation for the loop.
Hash iteration can fail if a GC collects a key in between
`hash-iterate-next` and `hash-iterate-key` (and similar). Use the
optional extra argument internally to detect and handle that case.
THe internal `assoc?` predicate is relatively expensive, so remove
redundant uses. Also, uses a cache to make it a little cheaper for
muliple uses of dictionary functions on a moderately sized list.
We have also removed ubsan flags from CPPFLAGS - since these are
compile flags, not preprocessing flags.
To pass the flags to rktio, we split the use of CFLAGS and CPPFLAGS
in rktio and ensure that flags in rktio do not update flags if make
is called in sub-make form (in a standard racket build, the rktio make
is called as a sub-make from the main racket make).
Fixes#2984
* Avoid export on variables in Makefiles
Instead pass the variables explicitly to the make call.
* Fix variable syntax
* Double quote variables
* Put ubsan back in CPPFLAGS
* Remove conditional assignment
This is not necessary if passing variables straight into sub-make
Instead of compiling the right-hand side of a `let-syntax`, interpret
using the schemify layer's safe-for-space interpreter. Similarly,
interpret the right-hand side of `define-syntax` when it is evaluated
during the enclosing module's expansion (compiling it for use when the
enclosing module is instantiated for expanding other modules).
This is an old idea, and it's effective in some cases: `racketcs -cl
racket/base` or `racketcs -cl racket` takes 20% less time with this
change. Various obstacles kept us from trying this earlier; most
significantly, the gap to finish the safe-for-space interpreter was
small enough to make it work. It's not clear that the result is an
improvement for `raco setup`, though.
The interpreter is not fast (it's about 1/4 the speed of the
traditional Racket interpreter), so there's room for improvement,
but even a slow interpreter pays off for building.
When recompiling modules from machine-independent mode to
machine-specific mode, dependencies of the module to be recompiled
must be checked explicitly, but using `module-path-index-resolve` is
not a good strategy, because it doesn't load transitive dependencies.
That can create a failure later when a transitive dependency is needed
for compilation and it isn't loaded because the dependent module was
already loaded.
Relevant to #2928
There could be many more functions that produce immutable strings
directly, and we want the default functions to do that in some future
language. For now, these three rae the most immediately useful for
avoiding unnecessary allocation in Racket CS.
Mostly reverts ee4ceb7ae4 and tries a different approach. The prevoius
approach did not handle the case where the module begin instantiated
was tiggered by a module at a different (higher) phase.
Closes#2981
Eliminates another ~300 instantiations from `racket -cl racket/base`.
small-scheme.rkt is no longer used in `racket/private`.
c.rkt was an accidentally-committe file.
Maybe there's a problem in `rktio_poll_add_connect` that I just can't
see, or maybe it's a Mac OS bug, but `rktio_poll_add_connect` doesn't
seem to reliably wake up the process when the TCP connection becomes
ready. Traditional Racket happens to avoid the problem by registering
the connection file descriptor with the semaphore table; doing that
for Racket CS also avoids the problem there.
Take advantage of new guarantees/support for immutable strings within
a Chez Scheme symbol representation. Consistent use of immutable
strings at the boundary avoids potential non-determinism.
This simplifies the expander logging in some places and adds logging for arming/disarming and scope changes (eg syntax-local-introduce) so the macro stepper can better track term identity.
This relies on corresponding changes to the racket/macro-debugger repo.
This breaks existing packages; see https://github.com/greghendershott/aws/issues/64
for an example.
Unfortunately, we probably have to live with this limitation of the
interface because of existing code.
This reverts commit 966399dec6.
Source locations are attached to functions for backtraces. With
traditional Racket, those source locations are connected to the
machinery of `current-write-relative-directory` and
`current-load-relative-directory` to avoid absolute paths, but that
machinery is difficult to integrate into the Racket CS compilation
model. So, since they're "just" for stack traces, save only a couple
of elements of the path.
Although some of them probbably do not matter (while some certainly
do), avoid various possible problems by always using a locally
determinsitic replacement for `gensym`.
Use newly added support for uninterned symbols (as opposed to gensyms)
in the Chez Scheme layer. Using uninterned symbols reduces
non-determinsitsm in the build.
The change to use Chez Scheme uninterned symbols exposed problems with
the way that Racket-level uninterned symbols (formerly implemented
with gensym) are handled in ".zo" files. The problem is that some
uninterned symbols are marshaled with `racket/fasl`, which is not
consostent with those that are marshaled by Chez Scheme's `fasl`. This
patch fixes those problems by ensuring that uninterned symbols are
always lifted to the level of a Chez Scheme `fasl` for a complete
linklet bundle.
Change the way names are generated for pieces of the implementation of
a keyword-argument function. These functions are not accessible as
values, so the names don't matter for printing a function, but the
names can show up in stack traces.
Building with `--enable-embedfw` creates executables that link
statically to the Racket runtime system, instead of linking to a
Racket framework, and that embed boot files in the case of Racket CS.
Executables built this way are meant to cooperate better with code
signing.
Related to #2910
Allow 0 as the port number in `ssl-listen`, and fix
`ssl-addresses` to work on a listener. Update a test
to use these features so that it won't conflict with
other tests that listen on TCP addresses.
Where true places are not supported, `place` is simulated
using Racket threads, and `place-kill` did not kill off the
simulated place well enough.
Relevant to #2930
A sequence of definitions (or `define-values`) at the end of
a module could get reordered. That only matters for the order
of checks for attempting to assign to a constant, though.
Suppress the compiler's conversion of statically obvious arity errors
to an expression that reports a custom error with the an S-expression,
since that interferes with Rumble's normalization of error messages.
If `struct dirent` has `namlen` field, define `HAVE_DIRENT_NAMLEN`.
If the field is instead `namelen`, define `HAVE_DIRENT_NAMELEN` (case of QNX).
Use this checks in `rktio_fs.c` and simplify `platform.h`.
Fix many incorrect arity declarations and actual arities in Racket CS,
and fix several incorrect arities in traditional Racket. Building
Racket CS now checks the information in "racket/src/cs/primitives"
against both Racket variants to make sure that they're all consistent.
Closes#2924
Racket CS did not support the optional second argument for
`continuation-prompt-available?`. Traditional racket did not produce a
sensible result for the prompt tag that is used to delimit a
composable continuation or in some cases for the default continuation
prompt tag.
configure scripts look for and read a local configuration file given by the
environment variable CONFIG_SITE. This can set variables such as prefix.
Racket’s build system was assuming that prefix would be set to NONE unless a
--prefix command line argument was given. But it could be set by a
CONFIG_SITE configuration file instead.
Hence, for in-place build add an explicit --disable-useprefix option, to
cause any prefix setting to be ignored, and use that in the top-level
Makefile.
Regenerate the configure scripts to get the updated code.
Revert the part of 39a96dd699 that hides the provenance of these
accessors. Although exposing the fact that the predicates are for
structure properties constrains some internal representations, that
constraint seems unlikely to matter, and exposing the procedures as
property predicates is more consistent with the documentation and the
implementation (especially for Racket CS).
Closes#2904
Improve gcc detection in configure script
We have been detecting gcc by the CC variable but this fails under
ubuntu for example where you might specify CC='gcc-8' to ./configure.
Also consider clang impersonator to be gcc.
Related to #2890
Support continuation-mark inspection proportional to the amount that
needs to be inspected, instead of having to build a list of length
propotional to the size of a continuation.
In Racket CS, use iteration to improve exception-handling chaining.
Traditional Racket already used similar functonality internally.
Add a special case for a real divided by a complex, and remove
(probably) misguided special cases for inexact zero real and imaginary
parts. These changes bring complex `/` further in line for Racket and
Racket CS.
Related to racket/typed-racket#868
The check for whether a provided identifier is syntax or a variable
cuold incorrectly inspect a module that is not yet prepared for the
relevant phase.
Include the defined name (in addition to the provided name) in the
result of `module->exports` and `module-compiled-exports` --- but for
backward compatibility, only if the new optional `verbosity` argument
is 'defined-names.
* Fix gen:dict methods for alists with duplicate keys
This fixes https://github.com/racket/racket/issues/2803.
Note that while this works, the implementation is not particlarly fast.
Especially for iterators.
* Fix tabs
* Make code more efficient
* Fix function definition order
* Import set library
* Preserve order
* Use immutable hash sets
* Remove unused import and remove duplicate checks
* Fix syntax error
