This commit adds `prop:struct-field-info` which is implemented to provide
static information about field names. The property is attached to all
struct types generated by `define-struct`.
The commit also modifies kernstructs to have the property.
Finally, the commit switches `struct-copy` to use the static field name
information when it's available.
It remains to change `contract-out` and `match`'s `struct*` to
recognize/attach this new property, but this could be done
separately in the future.
This PR is largely based on racket/racket#732, though the approach is slightly
different.
Although building with the wrong offsets is good enough to compile
the compiler to compile itself correctly, a broken intermediate
compilation can create confusion.
As the documentation says, `datum->syntax` should use
`datum-intern-literal`. That helps avoid syntax-object mutability, and
it increases sharing in compiled forms. The use of
`datum-intern-literal` got lost when the expander was rewritten in
Racket.
Relevant to #3245
The default package scope is determined by consulting the user-scope
configuration, then falling back to the installation-scope
configuration, then defaulting to user.
So, if you have a user-scope configuration of `default-scope`, it
doesn't matter what the configuration says in other scopes, which
means that the output of `raco pkg config` can be confusing. Extra
output in this commit is intended to make it less confusing.
Probably the original mistake here was allowing `default-scope` at a
scope-specific layer, instead of having it just as an installation
configuration (like `name`).
* Document liberal-defines? argument of generate-expand-context
* Add make-expression-transformer
* Ensure `block` expands in an expression context
Also refactors block to depend on racket/base and syntax/ modules rather
than private pre-base dependencies so it can use
`make-expression-transformer` and `generate-expand-context`.
* Ensure `local` expands in an expression context
* Add macro.rktl tests for block and local
An optimization relatively late in the BC bytecode compiler pipeline
was wrong for `begin0`. The transformation and bug must be a very old,
since it's intended to help the bytecode interpreter.
Thanks to Sage for reporting and Alexis for initial debugging.
When linking with libracket.a or libracket3m.a, librktio.a is needed.
(The instructions in "Inside" have apparently been wrong since rktio
was split out.)
The main (slightly) effective change here is to avoid disturbing loop
patterns within the Rumble layer's implementation.
Most of the commit is a commented out, updated version of the Scheme
implementation of MRG32k3a `random`. With the latest improvements for
unboxed floating-point arithmetic, performance is relatively good, but
it doesn't catch up to the C compiler's output. On an x86_64 MacBook
(i7 4870HQ) using LLVM or a Raspberry Pi 3 using GCC, it's about 50%
slower compared to C (in contrast to 300% slower before unboxing).
It's almost the same speed on a older x86_64 Linux machine (i7 2600)
using GCC. Where the C compiler wins, maybe it's due to the use of
SIMD instructions in the C output for x86_64 and Arm32. Switching to
the Scheme implementation of `random` would probably be fine, but
aisde from the satisfaction of being in Scheme, there's no reason to
pay the sometimes 50% penalty for now.
Caching compiled JIT fragments in a SQLite database did not turn out
to be a viable path, so remove partial support for it. JIT mode in
general is rarely a good option, but it's at least completely worked
out, so left in for now.
Update the Guide's performance section with current information for
Racket CS, and also document the Racket CS compilation mode and
inspection environment variables. Make a couple of environment
variables work more consistently: PLTDISABLEGC for CS and PLT_ZO_PATH
for BC.
When the runtime thread `touch`es a future that is blocked on an
atomic action (just as JIT compilation), the runtime thread would
eagerly run the action, but still leave the future on the
atomic-action queue. Atomic actions tend to be ok to run a second time
(including JIT compilation), so a problem may not show up immediately,
but a semaphore can get out of sync and cause problems later.
Change `fl->fx` to truncate as it converts, which is typically done
anyway by a machine instruction to convert from floating-point to
integer values. This makes `fl->fx` different from `inexact->exact`
or `fl->exact-integer`, but it brings BC and CS in line.
Follows Chez Scheme and Guile. Turns `(exp 10000.+0.0i)` into
`+inf.0+0.0i` instead of `+inf.0+nan.0i`, which is analagous to
the behavior for exact 0 in the complex part.
Fixes#3194.
If the callback takes too long to run, then a second copy was
scheduled --- which likely schedules a third copy, and so on. This
problem could lead DrRacket to get stuck in a GC cycle, for example.
In Schemify's lifting pass, avoid disturbing loop patterns that Chez
Scheme will recognize. Keeping the loops intact is helpful for
floating-point unboxing, where (currently) unbxoing is supported
across loop iterations but not across function boundaries in general.
When a `module` or `let-syntax` form is `expand`ed (i.e., when a
syntax object must be generated), then code is expanded to a syntax
object and then parsed again for compilation. In that second parse,
take advantage of the fact that the expression is already expanded,
which means that no new scopes or bindings need to be created.
Related to #3165
Two "self" module path indexes are treated the same on load when they
have the same resolved module name. Fix the serialization intern table
to take that into account and avoid some GC-based non-deterministism.
Related to #3165
On a second look, 6d06086dad unnecessarily duplicates work already don
by `copy-file` in most cases. For the case where `copy-file` was not
used, there was code to explicitly set an executable bit, but the new
approach is better and should be used just there.
Changes `_ptr`, `_list`, and `_vector` syntax to be more permissive.
Matching by symbol rather than binding is usually the wrong choice,
but the position where `i`, `o`, or `io` is recognized in `_ptr` and
company is not an expression position. Also, `i`, `o`, and `io` are
not otherwise bound. So, it's difficult to justify the current
behavior. If `_ptr` and company were defined today, we'd use keywords
like `#:i` instead of symbols like `i`.
A potential drawback of this change is that a `i`, `o`, or `io` might
be written with the intent of referring to a binding, and this change
makes such a reference silently allwowed instead of an errror. That
seems relatively unlikely, however, while having to avoid `i`, `o`, or
`io` as a binding is occassionally a pain --- something I've run into
once or maybe twice.
Closes#887
The Windows scheduler tends to work on 16ms bounaries, which
is not good for sleeping (as part of an animation, say) for
times near or less than 16ms. Change rktio to request more
fine-grained scheduling temporarily when trying to sleep for
a short time.
Extracting the name from a procedure involves `string->symbol` and
possibly some string parsing. Map the code object to the result
symbol to speed up multiple requests for the same code object.
* Ensure that the GMP ARM extra file has the correct extension
We hardcoded in some places the extension of the GMP ARM extra file
with .o, however the Makefile rule to build the file will use .lo is
--enable-shared is enabled. This commit fixes the discrepancy by not
hardcoding the extension anywhere.
Fixes#3176
* Avoid gnu make extension
* Replace spaces to conform with other lines
* Ensure we only add extension if file is needed
* Move the LTO definition to the configure.ac
* Carry the LTO variable into the Makefile
* Fix variable reference
* Add variable definition to gc2 Makefile
* annotate assignment
Concurrent lazy scope propagation had a (very unlikely) race condition
updating a syntax object. Ryan pointed out this problem as part of the
discussion for #3162.
Switch to using a single field for a syntax object's content and
propagations, so a CAS can be used to update the field. As a pleasant
side effect, this change tends to make syntax objects more compact.
Fix `ptr-ref` to properly handle the offset in a pointer for
specialized references, like extracting ` _double`.
Thanks to Laurent Orseau for the bug report and Jens Axel Søgaard for
intial debugging.
When a future triggers a write barrier, a lock is needed to prevent a
race between the future thread and other threads in the same place (on
most platforms).
Thanks to Dominik Pantůček for tracking down this bug.
Check `file-exists?` before passing the file on to `file-size` and/or
`open-input-file`. The latter led to confusing error messages, e.g.:
```
(file->string "DNE")
;; file-size: cannot get size
```
affects: `file->value` `file->list` `file->string` `file->bytes`
`file->lines` `file->bytes-lines`
The call to `random-sample/replacement` gets called when `(not
replacement?)` holds, while `random-sample/out-replacement` gets called
otherwise.
This is probably because the algorithm for the without replacement case
actually replaces parts of the sample, but that is different from the
meaning of a sample with and without replacement, which means with
possible duplicates or without.
This PR fixes four bugs:
1. Accessors are required at use-site in order to use `struct-copy`.
This PR removes that requirement since the information is already available in
struct-info. The following program used to fail prior the PR but will now pass.
```
(module a racket
(provide a)
(struct a (b)))
(require 'a)
(struct-copy a (a 1) [b 2])
```
2. `struct-copy` fails if the structure type transformer binding is renamed
(#1399). The following program used to fail prior the PR but will now pass.
```
(module struct racket/base
(provide (struct-out point))
(struct point (x y) #:transparent))
(require (rename-in 'struct [point point2d]))
(struct-copy point2d (point2d 1 2) [x 3])
```
3. With supertype, it's possible to construct colliding accessors,
causing `struct-copy` to update an incorrect field. The following program
produced incorrect outputs prior this PR but will now be correct.
```
(module a racket
(provide a)
(struct a (b-c) #:transparent))
(require 'a)
(struct a-b a (c) #:transparent)
(struct-copy a-b (a-b 1 2) [b-c #:parent a 10])
;; before the PR: (a-b 1 10), after the PR: (a-b 10 2)
(struct-copy a-b (a-b 1 2) [c 10])
;; before the PR: (a-b 1 10), after the PR: (a-b 1 10)
```
4. Similar to 3., prior this commit, it's possible to refer to a bogus field
name when supertype is present. The following program doesn't result in
a syntax error which is wrong. This commit fixes that.
```
(module a racket/base
(provide (all-defined-out))
(struct a (b-c) #:transparent))
(require 'a)
(struct a-b a (d) #:transparent)
(struct-copy a-b (a-b 1 2) [c 10])
```
The key idea is that the actual struct name (if the struct is created via
`struct` or `define-struct`) can be extracted from the name of struct predicate.
The actual field names then can be precisely extracted from accessors.
Note that struct-infos that are created manually by `make-struct-info`
didn't work with `struct-copy`. This PR didn't attempt to fix that because
it requires a significant change that would not be backward compatible with
the current struct info.
Instead of logging an error when the `openssl` module is loaded, defer
a complaint until procedures that would depend on the configuration is
called. Otherwise, errors can get printed in programs that depend on
the `openssl` library but do not always need OpenSSL support at run
time.
This is a backward-incompatible changed, but no packages currently
registered at pkgs.racket-lang.org refer to `ssl-dh4096-param-path`.
Providing `ssl-dh4096-param-bytes`, instead, avoids carrying along an
extra file with any stand-alone executable that depends on `openssl`.
Don't try to park values when allocating a weak box or pair in a
future thread, since that creates a race on the parking spaces. A
future thread can't run a GC, so it's doesn't need to park.
Touching a future in a future allocates a weak box, so this bug could
have been responsible for many crahses.
Related to #3145
Update GMP license statment following 392dc33ceb which comes from
a recent GMP version.
This does not affect the overall license situation of Racket BC,
which includes other LGPL v3 code, or of Racket CS, which does not
use GMP.
The `call-with-deep-time-limit` function in `racket/sandbox` expects a
subprocess to be removed from its custodian when the subprocess is
done. CS wasn't doing that at all, leaving custodian removal to a
finalizer. BC was doing delaying a remove until `subprocess-status` is
used (which happened to work for existing uses of
`call-with-deep-time-limit`, apparently.)
Relevant to #3140
When incremental mode is enabled, adjust garbage collection to avoid
promoting objects from the next-to-oldest generation into the oldest
generation. This change produces a good approximation to incremental
collection for game-like programs (although probably not server-like
programs with large, temporary jobs).
Fix error checking and reporting for position-based struct accessors
and mutators. Also, fix mutability recording for prefab structure
types that have auto fields.
Related to racket/typed-racket#902
Implement 'atomic-interior allocation and immobile cells using
`make-immobile-bytevector` and `make-immobile-vector`, which avoids
having to unlock through a finalizer.
Also, the Chez Scheme GC can now mostly mark a major generation,
instead of copying it, which can significantly reduce memory
use during a GC for an old, large heap (such as DrRacket's).
Show peak administrative as a parenthesized delta on peak space.
For BC, this extra delta is small, because BC compacts (instead of
copying) old-generation objects. For CS, the extra delta can large ---
typically an extra 50%, but potentially another 100% --- because a
full collection copies all old-generation objects.
Also, for BC, fix cumulative-allocation reporting to include child
places.
Extract invert_limb code for ARM from GMP 6.2.
In order to check for thumb mode availability use defined macro `__thumb__`, which in turns requires us to process the source file with `gcc` instead of `as` in order to access the preprocessor - tested with `clang` as well.
Fixes#3050
The Chez Scheme change avoids a leak while collecting in counting
mode, which is used by Racket's memory-accounting mode.
Also, add a small repair for 4256214981.
GitHub switched URLs for tarballs, redirecting to the new one with
a 302. However, old versions of Racket don't follow redirects in
`raco pkg install`, so they broke (before 6.3). Using the new URL
should work for everyone.
Reported by @greghendershott.
Also `--include-deps`, which support the creation of a catalog archive
that is restricted to a specific set of packages. Also
`--fast-file-copy`, which is usefl for speeding up a pipeline of
archiving (helpful to pkg-build).
Chez Scheme now supports a single-pass combination of `collect` and
`compute-size-increments`, which makes a GC with accounting about
twice as fast. Meanwhile, other GC improvements reduce non-accounting
full-collection times by 10-20%.
Much of the GC implementation is now generated from a "Parenthe-C"
description, so update the bootstrap process for that step.
This fixes a major issue in arm 32bits, detected by ubsan, where
conversion of type to milliseconds results in overflow.
```
rktio_time.c:92:21: runtime error: signed integer overflow: 1584975753 * 1000 cannot be represented in type 'long int'
```
GCC 9.3 things that `h2` might reach line 385 uninitialized.
```
./hash.c:385:14: warning: 'h2' may be used uninitialized in this function [-Wmaybe-uninitialized]
385 | h = (h + h2) & mask;
| ~~~^~~~~
```
A Chez Scheme garbage collection involves a rendezvous among threads
that are used to implement places and futures (and potentially other
things that create Chez-level threads). The thread that is used to
drive the garbage collection was not formerly specified, and callbacks
like the GC icon in DrRacket can only be run in the initial thread.
When a major collection was run in a non-initial thread, the callback
was simply skipped.
The Racket branch of Chez Scheme now drives a collection in the
initial thread whenever that thread is active. In Racket CS, the
initial thread can be inactive if it's waiting for external events,
and a place can meanwhile trigger a GC. Now, when the Racket CS GC
callback is called for a major collection in any thread other than the
initial one, it defers the major collection to an asynchornous
callback in the main thread (and meanwhile performs a minor
collection).
So, a major collection might now be delayed by just a little while if
the main thread is inactive, such as when it's waiting for external
events, but callbacks like the GC icon in DrRacket will be reliably
invoked for major collections.
It's still unclear what the specification of syntax->string regarding whitespace
before closing parens should be. The implementation also has not dealt with
the issue. This PR therefore removes whitespace before closing parens from
the tests.
Once the discussion at
https://github.com/racket/racket/issues/3071#issuecomment-601984438
has reached a conclusion and implemented, we can add these tests back.
The `call-in-continuation` function generalizes applying a
continuation to values by accepting a thunk that is called in the
restored continuation. In other words, insteda of having to use the
pattern
((call/cc (lambda (k)
.... (set! saved-k k) ...
(lambda ()
original-result))))
...
(saved-k (lambda () new-result))
The extra call and thunk on the capture side can be omitted:
(call/cc (lambda (k)
.... (set! saved-k k) ...
original-result))
...
(call-in-continuation saved-k (lambda () new-result))
At the Chez Scheme level, a `call-in-continuation` in tail position
within a function can avoid forming a closure for its second argument.
The `call-in-continuation` function at the Racket CS level doesn't yet
provide that benefit.
The `call-in-continuation` operation is called `continuation-slice` in
Feeley's "A Better API for First-Class Continuations".
Expansion of a procedure with keywords is quadratic due to generating
a nested sequence of `let`s, but speed it up by roughly a constant
factor by using a dintinct symbol for each nested layer.
Related to #3070
Using `call-in-continuation` to apply a thunk within a continuation
slightly simplifies and speeds up parts of the implementation of
delimited continuations.
The expansion of `struct` created far too much code to parameterize
`struct-field-index`, making expansion of a `struct` form with just
100 or 200 fields take a noticeably long time to expand.
PR #2678 unintentionally removed this attribute, but it was used
at least by "collections-lib" and "static-rename-lib".
cc @sorawee @lexi-lambda @jackfirth @rmculpepper
GitHub's CDN seems to have recently started returning the `Location` header for a redirect with a lowercase `l`, which breaks the redirect logic. The HTTP spec says that header names are case-insensitive, so we need to look for either version.
Make `register-process-global` check for byte strings, and avoid
retaining the byte string that it's given (in case that changes, for
example).
Closes#3053
Building glib-specific support into the main Racket executable is
unsatisfying, but it's consistent with Racket BC, and the alternative
is especially tedious to deal with places and namespaces and
allocation.
Combined with a schemify improvement that inlines imported predicates
to expose the record-type test to Chez Scheme, this chage enables
cptypes to prune useless inlined `wrapped-object` selector branches.
That improvement, in turn, reduces code size and redundant checks on
objects that have contracts.
When a `lambda` form is marked as a method (for arity error reporting)
through a property, translate that to a static flag on the procedure,
instead of a call to `procedure->method`.
The only way we have to attach static information is through the
procedure name, so the encoding already in place for "no name" and
"path-based name" is extended to support a method flag.
Since procedures have names that are attached more directly by the
schemify pass, remove simple `let-values` forms wrapping procedures.
This shortcut improves the result of the lifting pass in some cases.
Record types with unnamed fields can be significantly more compact,
excdeption in combination with the constrain the the fields all
contain Scheme objects. Saves 2% for DrRacket's initial footprint.
We had to give back some space savings to avoid potential problems
with allocating in the event callback and running out of stack
space[?] when passing an argument. We get to keep most of the
improvement though.
Fix a regression in 712494312a, and change other other two printers to
be more consistent for
#lang racket
(struct s () #:transparent)
(define a (s))
(pretty-print (list (cons a 0) (cons a 0)))
A regexp can match while having sub-patterns that are not used in
the match. In regexp-replace, the "insert" argument can refer to
these unused sub-matches. If a function is given for the "insert"
argument, it will receive these unused submatches as #f values.
If a string or byte-string is given, then any reference to an
unused sub-match (by means of a back-reference) is treated as the
empty string or byte-string.
Fixes#3032.
When logging GC debugging, a pecentage after the time for a GC reports
what fraction was extra steps after GCing proper, especially the extra
step of memory acounting when that is enabled.
Also, avoid Chez Scheme gensyms even more. Otherwise, using low-level
facilities like `dump-memory-stats` can force the unique name of a
gensym used for a structure type, which causes it to be permanent,
which could be exactly what you don't want when debugging a
memory-rentention problem.
In `compiler/private/mach-o` --- which is reachable via `racket` due
to being a dependency of `setup/dirs` --- delay the call to
`cross-system-library-subpath` until needed.
Fixes optimization for an expression like
(define (f x)
(lambda (y)
(letrec ([recursion (f x)])
(+ x y))))
by adjusting the inlining hueristic to support less inlining on a
second pass of `letrec` right-hand sides.
Closes#3027
Provide `vm-primitive` and `vm-eval` to regularize access to VM-level
primitives. Document some of the issues for interoperating across the
Racket and (Chez Scheme) VM layers.
The library could have been implemented with just `compile-linklet`
and `instantiate-linklet`, but using an underlying `primitive-lookup`
function is a little nicer.
Produce 0, -1, or out-of-memory for bignum shifts. For large fixnum
shifts, check memory limits.
The repairs are mostly for Racket CS, but traditional Racket
incorrectly reported out-of-memory for 0 shifted by a positive bignum.
A more compact representation for return points tends to reduce code
by about 10%. For DrRacket, that translates to a 5% decrease in
overall footprint.
Explain how to deal with the "racket-lib" that is normally included
with a source distribution, but that doesn't have dependencies
specific to Windows or Mac OS.
Since schemify adds its own checking for procedureness in an
application, always compile the application as unsafe at the Chez
Scheme level. This simple change saves about 5% in code size for
DrRacket, which is a 1-2% footprint saving overall.
An improvement to Chez Scheme allows more function from the Rumble and
other built-in layers to be inlined into compiled Racket code, and a
new `$app/no-inline` primitive enables improved control over how slow
paths are integrated.
The pretty printer and built-in printer for traditional Racket did not
consistently provide the current quoting mode while checking for
unquoting and cycles. All printers, including the Racket CS printer,
are improved for a structure type that has
`prop:custom-print-quotable` as 'always, in which case we know that
unquoting- and cycle-checking time that the components will be in
quoted mode.
The pretty printer also made three passes through a value to check for
cycles, compute cycles, and compute unquotes, and those are now fused
into a single pass like the Racket CS printer. The built-in printer
for traditional Racket still makes up to two passes, but it now
behaves more like other printers by recurring immediately on nested
calls via `prop:custom-write` instead of accumulating them for after
the `prop:custom-write` callback returns.
The documentation clarifies that synthesizing new values during
printing can interefere with cycle checking and unquoting, but the
printers now react to that behavior more consistently.
With recent improvements, the run-time performance of vector-stencil
HAMTs for immutable hash tables seems close enough (on
microbenchmarks) to the Patricia-trie implementation to be worthwhile,
since they use less memory. Performance remains better in most cases
than the traditional Racket implementation.
The table at the end of this message summarizes relative performance
on microbenchmarks. Overall, though, immutable hash-table operations
are already so fast that these difference very rarely translate to
measurable differences in overall run times --- not even for the macro
expander, which relies heavily on immutable hash tables to represent
scope sets.
Stencil-vector HAMTs tend to take about 1/3 the space of Patricia
tries, and those space savings can turn into run-time improvements in
applications by reducing GC time. I've observed a 10% reduction in
compile time for some programs. When building a full Racket
distribution, run time shrinks by about 2 minutes out of 80 minutes,
probbaly because just average memory use goes down by 10%. DrRacket's
initial memory footprint goes down by about 37M out of 657M (a 5%
savings).
Mincrobenchmark relative performance, normalized to previous Racket CS
implementation (measured on 2018 MacBook Pro, 2.7 GHz Core i7; Chez
Scheme can substitute POPCNT instructions at link time):
patricia = previous Racket CS implementation as a Patricia Trie
stencil = new Racket CS implementation as a stencil-vector HAMT
racket = traditional Racket implementation
patricia stencil racket
set-in-empty:eq#t: ==| ==| ==|=
set-many:eq#t: ==| ==|== ==|========
set-many-in-order:eq#t: ==| ==| ==|====
set-same:eq#t: ==| == ==|=
set-in-empty:eq: ==| == ==|=
set-many:eq: ==| ==|== ==|========
set-many-in-order:eq: ==| ==|= ==|=====
set-same:eq: ==| == ==|=
set-in-empty:eqv: ==| ==| ==|==
set-many:eqv: ==| ==|== ==|=========
set-many-in-order:eqv: ==| ==|= ==|=====
set-same:eqv: ==| ==| ==|=
set-in-empty:equal: ==| ==|== ==|===
set-many:equal: ==| ==|== ==|=====
set-many-in-order:equal: ==| ==|= ==|===
set-same:equal: ==| ==|= ==|===
ref:eq#t: ==| ==| ==|=
ref-fail:eq#t: ==| ==| ==
ref:eq: ==| ==| ==|=
ref-fail:eq: ==| ==| ==
ref:eqv: ==| ==| ==|====
ref-fail:eqv: ==| ==| ==|
ref:equal: ==| ==| ==|===
ref-large:equal: ==| ==| ==
ref-fail:equal: ==| ==| ==|===
ref-large-fail:equal: ==| ==| ==
removes:eq#t: ==| ==|=== ==|===========
add+remove:eq#t: ==| ==|= ==|=======
removes:eq: ==| ==|==== ==|============
add+remove:eq: ==| ==|= ==|=======
removes:eqv: ==| ==|=== ==|=============
add+remove:eqv: ==| ==| ==|========
removes:equal: ==| ==|== ==|=======
add+remove:equal: ==| ==|= ==|======
iterate-keys:eq: ==| ==| ==|=
iterate-vals:eq#t: ==| ==|= ==|=
iterate-vals:eq: ==| ==|= ==|=
iterate-unsafe-keys:eq: ==| ==| ==|=======
iterate-unsafe-vals:eq#t: ==| ==| ==|
iterate-unsafe-vals:eq: ==| ==|= ==|
for-each:eq: ==| ==| ==|==========
subset-lil-shared:eq: ==| ==| ==|=
subset-lil-unshared:eq: ==| ==| ==|==
subset-lil-not:eq: ==| == ==
subset-med+lil-shared:eq: ==| ==|==== ==|=
subset-med+med-shared:eq: ==| ==|= ==|=
subset-big-same:eq: ==| ==| ==|===============
subset-big+lil-shared:eq: ==| ==|=== ==|====
subset-big+med-shared:eq: ==| ==|== ==|===
subset-big-unshared:eq: ==| ==| ==|==
This commit adds an (unused) implementation of immutable hash tables
for Racket CS that trades some run-time performance for an especially
compact representation --- similar to the traditional Racket
implementation of immutable hash tables. It uses a new "stencil
vector" datatype at the Chez Scheme level, which overlays the bitmap
needed for a HAMT node with the Chez-object type tag (and also
provides an update operation that avoids unnecessary memory work).
Compared to the current Racket CS implementation, the stencil-vector
HAMT implementation of an immutable hash table takes only about 1/3
the space on avergae, which translates to a overall 5% savings in
DrRacket's initial heap. It also makes a full Racket build slightly
faster by reducing avergage memory use by 5-10%.
But the run-time performance difference is significant, especially for
the `hash-keys-subset?` operation (at least in microbenchmarks), and
also for addition and iteration. Maybe there's an overall better point
that reduces memory use of the current Patricia trie implementation
without sacrificing as much performance.
Besides the benchmarks and stencil-vector HAMT implementaiton, there
are small changes to the way hash tables cooperate with `equal?`,
which makes it a little easier to plug in different implementations.
Building with shared libraries is not currently supported, because the
Chez Scheme build is not set up to work in that mode, and because
"stand alone" executable handling at the Racket level does not support
Racket CS shared libraries.
Also, there's no benefit to shared libraries. Racket executables get
the benefit of sharing because they all run through the same
executable. Meanwhile, there's not (yet?) a supported C API to make
something like "libracketcs.so" useful.
Related to #2993
Adjust part of the internal scheduling protocol to make a retry
callback generated by another callback that sets up the retry. This
helps clarify the protocol and avoids allocating a closure that is
rarely used.
Make Racket CS consistent with traditional Racket in the way
`chaperone-evt` on a thread hides threadness, etc.
Hiding properties like threadness is not ideal and does not seem
entirely consistent with `chaperone-of`, but allowing things like
threads and semaphores to be chaperoned creates non-trivial expense
internally. It would have been better to have event constructors for
threads and such to (and then the consyructed events could be
chaperoned without imposing a cost on the original data structure).
Force inlining of value and key accessors. Keeping the `define` with a
loop body --- that is, not making the individual function a macro ---
allows the Rumble `define` to avoid a closure allocation for the loop.
Hash iteration can fail if a GC collects a key in between
`hash-iterate-next` and `hash-iterate-key` (and similar). Use the
optional extra argument internally to detect and handle that case.
THe internal `assoc?` predicate is relatively expensive, so remove
redundant uses. Also, uses a cache to make it a little cheaper for
muliple uses of dictionary functions on a moderately sized list.
We have also removed ubsan flags from CPPFLAGS - since these are
compile flags, not preprocessing flags.
To pass the flags to rktio, we split the use of CFLAGS and CPPFLAGS
in rktio and ensure that flags in rktio do not update flags if make
is called in sub-make form (in a standard racket build, the rktio make
is called as a sub-make from the main racket make).
Fixes#2984
* Avoid export on variables in Makefiles
Instead pass the variables explicitly to the make call.
* Fix variable syntax
* Double quote variables
* Put ubsan back in CPPFLAGS
* Remove conditional assignment
This is not necessary if passing variables straight into sub-make
Instead of compiling the right-hand side of a `let-syntax`, interpret
using the schemify layer's safe-for-space interpreter. Similarly,
interpret the right-hand side of `define-syntax` when it is evaluated
during the enclosing module's expansion (compiling it for use when the
enclosing module is instantiated for expanding other modules).
This is an old idea, and it's effective in some cases: `racketcs -cl
racket/base` or `racketcs -cl racket` takes 20% less time with this
change. Various obstacles kept us from trying this earlier; most
significantly, the gap to finish the safe-for-space interpreter was
small enough to make it work. It's not clear that the result is an
improvement for `raco setup`, though.
The interpreter is not fast (it's about 1/4 the speed of the
traditional Racket interpreter), so there's room for improvement,
but even a slow interpreter pays off for building.
When recompiling modules from machine-independent mode to
machine-specific mode, dependencies of the module to be recompiled
must be checked explicitly, but using `module-path-index-resolve` is
not a good strategy, because it doesn't load transitive dependencies.
That can create a failure later when a transitive dependency is needed
for compilation and it isn't loaded because the dependent module was
already loaded.
Relevant to #2928
There could be many more functions that produce immutable strings
directly, and we want the default functions to do that in some future
language. For now, these three rae the most immediately useful for
avoiding unnecessary allocation in Racket CS.
Mostly reverts ee4ceb7ae4 and tries a different approach. The prevoius
approach did not handle the case where the module begin instantiated
was tiggered by a module at a different (higher) phase.
Closes#2981
Eliminates another ~300 instantiations from `racket -cl racket/base`.
small-scheme.rkt is no longer used in `racket/private`.
c.rkt was an accidentally-committe file.
Maybe there's a problem in `rktio_poll_add_connect` that I just can't
see, or maybe it's a Mac OS bug, but `rktio_poll_add_connect` doesn't
seem to reliably wake up the process when the TCP connection becomes
ready. Traditional Racket happens to avoid the problem by registering
the connection file descriptor with the semaphore table; doing that
for Racket CS also avoids the problem there.
Take advantage of new guarantees/support for immutable strings within
a Chez Scheme symbol representation. Consistent use of immutable
strings at the boundary avoids potential non-determinism.
This simplifies the expander logging in some places and adds logging for arming/disarming and scope changes (eg syntax-local-introduce) so the macro stepper can better track term identity.
This relies on corresponding changes to the racket/macro-debugger repo.
This breaks existing packages; see https://github.com/greghendershott/aws/issues/64
for an example.
Unfortunately, we probably have to live with this limitation of the
interface because of existing code.
This reverts commit 966399dec6.
Source locations are attached to functions for backtraces. With
traditional Racket, those source locations are connected to the
machinery of `current-write-relative-directory` and
`current-load-relative-directory` to avoid absolute paths, but that
machinery is difficult to integrate into the Racket CS compilation
model. So, since they're "just" for stack traces, save only a couple
of elements of the path.
Although some of them probbably do not matter (while some certainly
do), avoid various possible problems by always using a locally
determinsitic replacement for `gensym`.
Use newly added support for uninterned symbols (as opposed to gensyms)
in the Chez Scheme layer. Using uninterned symbols reduces
non-determinsitsm in the build.
The change to use Chez Scheme uninterned symbols exposed problems with
the way that Racket-level uninterned symbols (formerly implemented
with gensym) are handled in ".zo" files. The problem is that some
uninterned symbols are marshaled with `racket/fasl`, which is not
consostent with those that are marshaled by Chez Scheme's `fasl`. This
patch fixes those problems by ensuring that uninterned symbols are
always lifted to the level of a Chez Scheme `fasl` for a complete
linklet bundle.
Change the way names are generated for pieces of the implementation of
a keyword-argument function. These functions are not accessible as
values, so the names don't matter for printing a function, but the
names can show up in stack traces.
Building with `--enable-embedfw` creates executables that link
statically to the Racket runtime system, instead of linking to a
Racket framework, and that embed boot files in the case of Racket CS.
Executables built this way are meant to cooperate better with code
signing.
Related to #2910
Allow 0 as the port number in `ssl-listen`, and fix
`ssl-addresses` to work on a listener. Update a test
to use these features so that it won't conflict with
other tests that listen on TCP addresses.
Where true places are not supported, `place` is simulated
using Racket threads, and `place-kill` did not kill off the
simulated place well enough.
Relevant to #2930
A sequence of definitions (or `define-values`) at the end of
a module could get reordered. That only matters for the order
of checks for attempting to assign to a constant, though.
Suppress the compiler's conversion of statically obvious arity errors
to an expression that reports a custom error with the an S-expression,
since that interferes with Rumble's normalization of error messages.
If `struct dirent` has `namlen` field, define `HAVE_DIRENT_NAMLEN`.
If the field is instead `namelen`, define `HAVE_DIRENT_NAMELEN` (case of QNX).
Use this checks in `rktio_fs.c` and simplify `platform.h`.
Fix many incorrect arity declarations and actual arities in Racket CS,
and fix several incorrect arities in traditional Racket. Building
Racket CS now checks the information in "racket/src/cs/primitives"
against both Racket variants to make sure that they're all consistent.
Closes#2924
Racket CS did not support the optional second argument for
`continuation-prompt-available?`. Traditional racket did not produce a
sensible result for the prompt tag that is used to delimit a
composable continuation or in some cases for the default continuation
prompt tag.
configure scripts look for and read a local configuration file given by the
environment variable CONFIG_SITE. This can set variables such as prefix.
Racket’s build system was assuming that prefix would be set to NONE unless a
--prefix command line argument was given. But it could be set by a
CONFIG_SITE configuration file instead.
Hence, for in-place build add an explicit --disable-useprefix option, to
cause any prefix setting to be ignored, and use that in the top-level
Makefile.
Regenerate the configure scripts to get the updated code.
Revert the part of 39a96dd699 that hides the provenance of these
accessors. Although exposing the fact that the predicates are for
structure properties constrains some internal representations, that
constraint seems unlikely to matter, and exposing the procedures as
property predicates is more consistent with the documentation and the
implementation (especially for Racket CS).
Closes#2904
Improve gcc detection in configure script
We have been detecting gcc by the CC variable but this fails under
ubuntu for example where you might specify CC='gcc-8' to ./configure.
Also consider clang impersonator to be gcc.
Related to #2890
Support continuation-mark inspection proportional to the amount that
needs to be inspected, instead of having to build a list of length
propotional to the size of a continuation.
In Racket CS, use iteration to improve exception-handling chaining.
Traditional Racket already used similar functonality internally.
Add a special case for a real divided by a complex, and remove
(probably) misguided special cases for inexact zero real and imaginary
parts. These changes bring complex `/` further in line for Racket and
Racket CS.
Related to racket/typed-racket#868
The check for whether a provided identifier is syntax or a variable
cuold incorrectly inspect a module that is not yet prepared for the
relevant phase.
Include the defined name (in addition to the provided name) in the
result of `module->exports` and `module-compiled-exports` --- but for
backward compatibility, only if the new optional `verbosity` argument
is 'defined-names.
* Fix gen:dict methods for alists with duplicate keys
This fixes https://github.com/racket/racket/issues/2803.
Note that while this works, the implementation is not particlarly fast.
Especially for iterators.
* Fix tabs
* Make code more efficient
* Fix function definition order
* Import set library
* Preserve order
* Use immutable hash sets
* Remove unused import and remove duplicate checks
* Fix syntax error
For a division c+di, when c is less than d, and when (/ d c) produces
infinity, then the sign on the imaginary part of the result was
backwards. For example, `(/ -1.0e-9-1.0e+300i)` produced `-0.0-0.0i`
instead of `-0.0+0.0i`.
Note: the traditional Racket executable continues to incorporate
LGPLv3 code and thus modifications to it must be released under
that license. However, all Racket code in this repository, as well
as the Racket-on-Chez excutable, are more permissively licensed.
Thanks in particular to @otherjoel and @zyrolasting for their work
on this project.
Commit 77023aeaba made the reference from a custodian-managed
value to the custodian weak, and that could cause a crash in
certain shutdown cases.
Relevant to #2867
Just bumping the version number here to make sure the improved
`call/cc` is used. The improvement saves about 10% on ctak by avoiding
a layer of closure allocation.
Refactor to move some composable-continuation support out of the
way of prompts, and add a shortcut for simple composition cases.
Also, fix stack traces with continuation barriers and composable
composition, which could show sections of a trace duplicated.
For Racket CS, weaken references from managed objects to managing
custodians. Otherwise, a custodian with any managed values cannot
be GCed. Also, fix `collect-garbage` call that is triggered by
a memory-limit shutdown to happen after the current thread
(likely shutdown) is swapped out.
For traditional Racket, fix custodian shutdown on a memory-limited
custodian so that it is unregistered as having a limit.
Lift and delay serialization for non-serializable literals when
using `compile`. Just `compile`ing such an expression is ok, but
it reports an error if there's an attempt to serialize (by printing)
the compiled value.
This improvment also brings Racket CS much more in line with
traditional Racket on the kinds of values that it is willing to
serialize. For example, non-prefab structures no longer serialize
(where deserializing in a new Racket run would produce an instance of
a distinct structure type). The exception type and error message also
now matches traditional Racket.
The improvement relies on a new argument to `s-exp->fasl` for handling
errors.
Also, adjust a memory-limit test that wasn't checking behavior as
intended and that wasn't consistent with a Racket CS improvement over
traditional Racket.
A base type of `_gcpointer` works ok for traditional Racket as a kind
of "maybe it's GCable, so treat it that way just in case". That
approach doesn't work for Racket CS, where `_gpointer` has to mean
"definitely GCable memory". Although the difference is unfortunate,
making Racket CS base `(_bytes o <n>)` on `_pointer` is consistent
with the way plain `_bytes` is like `_pointer` in Racket CS and like
`_gcpointer` in traditional Racket.
Some synchronization built into `close-output-port` seems unnecessary
on modern Windows (NT and up). An extra manager thread is needed for a
anonymous pipe for "write ready?" purposes, but not to buffer output.
The Windows documentation is not entirely clear on this point, but
experiments suggest that modern anonymous pipes behave in the obvious
way.
Meanwhile, adjust the io layer used by Racket CS with the
synchronization that would be needed for Windows 95. This adjustment
is questionable, because it doesn't seem likely that we'll ever go
back that far with Racket CS. But the rktio interface to support
flushing synchronization might somehow be needed in a future setting,
or mabe it will turn out that I'm wrong about pipe buffers.
Avoid a continuation frame and layer of thunks that was in place for
checking for breaks just after applying a continuation. Instead, we
install just the continuation marks and check for breaks before
actually jumping; the break checker can't tell the difference, since
marks are the only way for it to check the continuation. This
improvement cust about 40% of the time for simple continuation capture
and application.
A misplaced `wrap-evt` could allow the result from `sync` on
a log receiver to be an opaque event, instead of a vector.
In other cases, a differently misplaced `wrap-evt` could also cause an
internal instance of `control-state-evt` to not be unregistered
correctly.
The solution to both problems is to add a wrapper procedure to
`control-state-evt`.
Closes#2664
Repair problems with asynchronous callbacks for futures and for
foreign callbacks. Asynchronous callbacks are used for future "sync"
operations, like `hash-set!`, that run must in a place's main thread
(as of commit f574583907).
Separately, synchronization to clean up future threads used a `ping?`
flag in a backwards sense, and it also treated a record as a box.
These problems could cause place termination to hang.
Related to #2725
The `math` library relies on this working right, since MPFR is
normally not compiled as thread-safe.
Also, fix some locking/interrupt/atomicity problems with async
callbacks generally.
When `sync` or `place-channel-get` is used on a place channel whose
other end has been GCed, then the blocking thread should also be
GCable. The `sync` case didn't work because the implementation uses
`replace-evt`. Change `sync` so that it can recognize asynchronous
`replace-evt`s in the same way as semaphores and channels (which is
more than traditional Racket offers).
When a place terminates, it was directly accessing its parent's
custodian. Prior support for cross-place uses of a hash table
probably helped hide this problem previously.
Although extremely unlikely, it was possible for multiple Racket
threads operating on the same scopes to race on a multi-scope's table
mapping phase levels to scopes.
Also, for some some mutable hash tables that will be shared across
places as read-only in Racket CS, make sure they are definitely set up
for iteration.
An `eq?`-based hash table in the implementation of custodians was
still shared across threads.
Also, taking the global lock at the Rumble level did not disable
interrupts. Since sometimes the lock is taken with interrupts
disabled, threads could potentially deadlock by not having an order.
Fix the problem by disabling interrupts before taking the lock.
Use the pseudo-random generator API that is now available from Chez
Scheme. While the generator can be written in Scheme, the lack of
unboxed floating-point arithmetic unfortunately makes it about 6 times
as slow as a built-in implementation. That difference is significant
when `sync` uses `random` for fair scheduling.
The `time-apply` function was measuring thread time instead of proecss
time. While thread time would be more useful in many cases, it's meant
to report process time.
Mutable `eq?`- and `eqv?`-based hash tables were formerly guarded by a
lock that made them safe for Scheme threads (i.e., OS-level threads).
In particular, that futures could concurrently access hash tables. But
the cost of that lock appears to be too high for such a rarely-used
capability.
Switching `eq?`- and `eqv?`-based hash tables so that they're safe
only for Racket threads means that the lock on a hash table can be
much cheaper. A lock is still needed to because the Rumble layer adds
extra fields for iteration. In the specific case of `hash-ref` on
`eq?`-based tables, however, the lock can be ignored, which makes one
of the most common `hash-ref`s much faster.
Overall, `hash-ref` on a mutable `eq?`-based hash table is now 4-5
times as fast, which makes it about twice as fast as traditional
Racket's `hash-ref`. A `hash-set!` operation is about twice as fast as
before, which puts it on par with traditional Rackets `hash-set!`. The
`hash-ref` improvement makes `send` about twice as fast as before in
Racket CS, making it a little faster than traditional Racket.
Since futures can no longer concurrently access `eq?`- and
`eqv?`-based hash tables, they have to synchronize with the main
thread for access. Racket CS had avoided the "sync" action on futures
that traditional Racket sometimes uses, but this change introduces
sync actions to Racket CS, since it's appropriate for accessing
mutable `eq?`- and `eqv?`-based hash tables.
Adjust the internal engine protocol to avoid a jump from a starting
engine (representing a thread) to a scheduler outside of an engine
to a target engine (for a swapped-ni thread); instead, jump from the
first engine to the target, effectively running the scheduler within
the starting engine's context.
In a file-stream output port or TCP output port, when flushing
encounters an error, consistently discard bytes in the buffer. This
isn't the obviously right choice, but otherwise a future flush attempt
(including one triggered by trying to close the port or one triggered
by a plumber) will likely just fail again, which is probably worse
than dropping bytes.
Also, fix related problems/inconsistencies.
Overall changes:
* For traditional Racket, discard bytes in a TCP port when flushing
fails.
* For Racket CS, discard bytes in file-stream and TCP output ports
when flushing fails.
* For traditional Racket, when a file-stream port flush is
interrupted by an asynchronous break, *don't* discard buffered
bytes.
* For Racket CS, don't register TCP ports with the current plumber.
When the original compiler handler is called with a true second
argument, then the resulting module is not serializable. Improve
detecting and reporting of the misuse.
The error is phrase in terms of linklets, which is not ideal, but
that's the level where the error can be detected. Abusing the original
compile handler in this way is not easy, though, so maybe this
improvement is enough.
Adds an additional line to the error message that is raised when a
required module provides a binding that is already provided by another
required module. The additional line displays the name of the first
module that provides the binding.
The error before this change:
tmp/c.rkt:4:9: module: identifier already required
at: x
in: "b.rkt"
location...:
tmp/c.rkt:4:9
and after:
tmp/c.rkt:4:9: module: identifier already required
at: x
in: "b.rkt"
also provided by: "a.rkt"
location...:
tmp/c.rkt:4:9
Add text of MIT and Apache v2 licenses.
Add initial CONTRIBUTING.md file which specifies contribution license.
Add COPYRIGHT.txt file which specifies the license and lists some
external components.
The LGPL license stays in its current location to avoid having to
modify the build right now.
The Mac OS 10.15 headers include a `#pragma` just before the closing
`;` of a `struct` declaration. That confuses poor xform. Handle this
special case by detecting it and swapping the order of the `#pragma`
and `;`.
If a source file name lacks an extension, then the pkg-step would get
confused trying to convert a ".zo" name back to a source name. The
original name is not really needed, anyway.
When a reference to a local variable is updated with the scopes of its
binding in a fully expanded program, remove the syntax-original
property if the original reference had macro-intrudction scopes.
Closes#2820
Trying to be more helpful about the thread running an `unsafe-poller`
callback gets in the way of making the process sleep when multiple
threads are blocked on unsafe pollers.
Closes#2833
Prune some `with-continuation-marks` forms that aren't observable
(because the body is simple enough that it won't inspect marks). More
significantly, specialize `with-continuation-marks` forms to indicate
when the current frame is known to have no marks and to indicate
when tthe key expression is known to produce a non-impersonator.
Since `#%app` (used where an applicable structure might show up)
injects its own `procedure?` test and makes sure that that a procedure
is returned to the function position of the application, use `#3%$app`
to make Chez Scheme suppress a redundant `procedure?` check for the
appliction.
It's not clear that a thread can be descheduled without the current
thread's work counting as progress, but a descheduled thread certainly
shouldn't coun as a no-progress scheduled thread.
Only one instance of each callback is needed. Allowing them to pile up
is inefficient, and possibly it can trigger a reaction that causes
even more to pile up.
(for ([x (in-value 1 2)]) x)
should raises a run time error, not a syntax error.
Fix similar error in other in-something macros.
Fix name of in-directory, when used as a function outside a for.
Unfortunately, MZ_NORETURN spec is causing a few problems - see #2808
It would be great to fix these but due to lack of time, this is a
workaround that should keep things working until all supported
configurations accept MZ_NORETURN properly.
Use `call-consuming-continuation-attachment` to implement
`with-continuation-marks`, because that avoids duplicating
a set of checks when in tail position.
Looking at `bytes-allocated` usually works, but sometimes lets
memory use spiral out of control. Looking at `current-memory-bytes`
is more reliable, but still makes peak memoy use too high.
Combining those values doesn't limit the peak well enough.
Try the more obvious (in retrospect) approach of comparing
`bytes-allocated` changes and `current-memory-bytes` changes
separately, and triggering a GC if either grows enough.
Commit 1811193285 caused Racket CS to have much higher peak memory
use. Adjust the heuristic again to trigger a major GC when the
`current-memory-bytes` value is the post-GC `bytes-allocated` plus the
post-GC `current-memory-bytes`, which means waiting until
another `bytes-allocated` bytes are allocated (instead of waiting
until the number of newly allocated bytes also catches up to overhead,
such as unserused pages due to locked objects).
Implement a parameter as a Chez Scheme wrapper procedure,
instead of an applicable record. A wrapper procedure can be
applied more directly, saving 10-20% of the time for some
parameter lookups.
The grammar for pregexps includes:
| \p{‹property›} Match (UTF-8 encoded) in ‹property›
| \P{‹property›} Match (UTF-8 encoded) not in ‹property›
and <property> is defined as:
‹property› ::= ‹category› Includes all characters in ‹category›
| ^‹category› Includes all characters not in ‹category›
That is to say, there are two independent ways to negate one of
these character classes. The Racket implementation of regexps
(as opposed to the C implementation) does not recognize negated
categories. This PR fixes that.
Some parts of the GC meant to traverse all objects on a page of
'atomic-interior or 'interior objects used "<" to detect the end of
the page, but "<=" was needed. As a result, things could sometimes go
wrong for the last object on a page for platform and size combinations
where the last object ends exactly at the end of the page.
This change consistenly computes the iteration end in a way that makes
both "<" and "<=" work.
Using MPFR bindings from the `math` library could trigger a problem
(but it's difficult to provoke the problem in a reasonably small
example --- difficult enough that I couldn't do it).
Use `current-memory-bytes` instead of `bytes-allocated` to determine
major GCs, because the latter doesn't include enough (perhaps missing
finalized values). For example, the repair avoids unbounded memory use
from
(let loop ([i 0])
(malloc 6400 'atomic-interior)
(loop))
due to finalizers that pile up faster than they are run.
When a linklet is too large to pass to Chez Scheme whole, then
names for the procedures that are individually compiled need to
be extracted from 'inferred-name for reference in the wrapper.
Closes#2787
Enables native dark mode UI elements in macOS 10.14 and above. Adds the
'NSRequiresAquaSystemAppearance' key with a value of 'false' to the
Info.plist file, while allows UI elements to match the system theme even
when not building against the latest SDK.
When opening the write end of a fifo that doesn't have a reader
already, the old implementation could allow writing bytes that are
discarded. This new implementation uses a blocking `open` in a
`pthread`, and that way the write routines know whether the stream is
ready for writing or not.
The difference is visible in the Racket API in a two places:
`subprocess` needs to wait until a fifo writer is connected before
attempting to dup the corresponding file descriotor, and more
generally a use of `unsafe-port->file-descriptor` needs to wait. The
former blocking operation is now build into `subprocess` (and
documented), but the burden is place on callers of
`unsafe-port->file-descriptor` to wait is necessary.
The new `port-waiting-peer?` predicate exposes the waiting state,
while `sync` is sufficient to wait for a peer.
Closes#2784
On platforms other than Windows and MacOS, locale encoding (inclduing
path <-> string conversion) opened a converter for each separate
operation. That can be slow on some OSes, so cache converters used for
locale conversions.
Relevant to #2781
Allowing #f as an allocator avoids problems composing `allocator` with
foreign-function lookup where failure is anticipated and implemented
as #f. For example, `g_settings_new` in the "gui-lib" package's
"mred/private/wx/gtk/gsettings.rkt" can be #f if the libgio libray is
too old, in which case there won't be an attempt to use
`g_settings_new`.
In some cases, (vector x 2 3 3 3) was pretty-printed as "(vector x 2 3)" when
print-vector-length was enabled.
Also print "(fxvector)" instead of "(fxvector )".
This repair affects DrRacket and xrepl after `enter` so that a
`require` in the context of a module namespace is resolved relative to
the module's path (as it did in the old expander, before v7.0).
Closesracket/drracket#276
Fix the default argument in `syntax/template`, where cons-proc-stx should be a
syntax object reflecting a procedure, not a procedure itself.
Fix test case for `syntax/template`, where restore-stx should be a syntax object
reflecting the restore procedure, not the proceure itself.
Closes#2745
For now, the operation is implemented on mutable tables in Chez
using a combination of hashtable-contains? and hashtable-cell.
A more efficient version will require modifying Chez.
Compute the list (at compile time), instead of using a literal copy of
the output at one point.
Also, adjust the documentation to explain extra guarantees provided by
`make-known-char-range-list`.
Closes#2757
Using GC_free() on runstacks was a way of reducing leaks with
conservative GC, but the implementation of prompts for delimited
continuations evolved in a way that is incompatible with using
GC_free(). The recent change to make certain runstack links weak
particularly exposed the mismatch.
Fixes#2748
Steps to create this certificate:
openssl genrsa -out key.pem 2048
openssl req -new -sha256 -key key.pem -out csr.csr
openssl req -x509 -sha256 -key key.pem -days 3650 -in csr.csr -out certificate.pem
cat key.pem certificate.pem > test.pem
When filling out the information for the certificate, keep the same
details - some tests depend on it.
Bring CGC and 3m in line by breaking weak links in CGC only after
ready level-1 finalizations (which correspond to will executors) have
been queued. The chage delays clearing of the weak link until after
the finalizer is run.
by `contract-out` instead of `->`
This seems to save about a second of startup time in (non-cs) DrRacket
and about .1 seconds in drracketcs.
The code is also theoretically more easily inlined. Still not easy enough, however.
In a pattern like
(let loop ([l l])
(define v (syntax-e l))
(cond
[(null? v) 'done]
[else
(loop (datum->syntax #f (cdr v)))]))
the running time was O(N^2) for a list of syntax objects of length N.
That pattern is relevant for traversals that use `syntax-case` like
(let loop ([l l])
(syntax-case l ()
[(a . b) (loop #'b)]
[() 'done]))
Avoid bad behavior by recording in an weak hash table certain pairs
that were previously been produced by `datum->syntax` internally so
that they can be used as-is.
Mainly for cross-compilation with a Windows target, the Racket CS
build process may need to run `lz4` as an external program. In that
case, complain when `lz4` isn't found, instead of letting `system*`
complain about getting `#f`.
Change the condition for filtering leaf contracts via `eq?`.
Before, we looked for flat or chaperone contracts.
After, look for flat or trusted contracts. So:
1. untrusted chaperones with side effects that are unsafe to drop are not
dropped, and
2. impersonator contracts can now be dropped (object/c, recursive-contract)
When compiling with -Werror (./configure CFLAGS="-Werror" ...),
the test to see if mbsrtowcs exists
failed with pointer type of incompatible type, is char **, should be
const char **. It would proceed to assume mbsrtowcs didn't exist.
Ensures proper noreturn annotations for error functions. Implemented
cross-platform unreachable annotation. No warnings in tested clang or
gcc with default flags. Tested as well on MacOS and Windows.
The schemify pass collects known-value information as the first step
of processing a linklet body, but the main pass to process the linklet
body may simplify it in a way that exposes new information. For
example, in
(define (call) (values 1 2))
(define-values (x y) (call))
the main pass will inline `call` and expose the fact that `x` and `y`
are always 1 and 2, respectively.
Adjust schemify to inspect the simplified form of a definition and
potentially add new information to known-value information, which is
useful later in the ame linklet body and also as cross-module
information.
Like other optimizations that schemify duplicates, constant folding
helps support cross-module optimization. Related "no-prompt"
declarations for primitives can reduce `call-with-module-prompt`s in
schemified output, too, which can interfere with Chez Scheme's
optimizer.
This is probably related to #2712.
It's the only occurrence of SCHEME_NO_EXN pointing to the fact that
this is an historical artifact that can be removed.
Recognize `(define-values (id ...) (values rhs ...))` and split to
multiple `define`s after simplifying the right-hand side of
`define-values`. Also, don't split if a define variable is referenced
too early.
Adding return statement where it doesn't exist, causes problems with
functions marked no return so it should be avoided.
Related to #2709 - with this PR, clang won't complain any longer
about issues with xform generated sources about functions marked
no return that do indeed return.
When `set!` is used to tie a recursove knot in a flattened linklet,
the expander can recognize that pattern and not complain that a
mutable variable might be shared across places. Improving that
inference means fewer `++global-ok` flags in Makefiles.
This commit also avoids single-quote as an escaping form in a
makefile, which doesn't work with nmake on Windows.
Continue to build the bundled-with-Chez zlib and lz4 by default, but
support `--enable-libz` and `--enable-liblz4`, and turn those on when
the Chez Scheme directory doesn't have bundled versions. That should
make things build right for distributions where repackaging
dependencies is disallowed or discouraged.
The Chez Scheme build process now create an archive instead of linking
"kernel.o". Adjust the Racket CS build to use archives instead of
"kernel.o".
Also, modernize the Racket build's use of `ar`. Using the flags `rc` by
default (instead of `ruv`) should avoid the need for `nicear`.
If GUN ar is configured for deterministic mode by default, then
ar: `u' modifier ignored since `D' is the default (see `U')
prints. Suppress that to avoid stderr output.
Checking for directories when searching for a collection can be
especially slow with the Windows filesystem, but Windows also supports
filesystem-change events. So, cache information about the existence of
paths, relying on filesystem-change events to detect with the cache is
out of date.
Related to racket/drracket#281
The prompt installed for an escape continuation or continuation
barrier is never used to delimit a captured continuation, so the
return from the continuatiton can be streamlined slightly.
The benefit is very small (but detectable in the macro expander's use
of barriers). There's an opportunity to use `call/1cc` instead of
`call/cc`, but that change does seem to help --- neither Chez Scheme's
current `call/1cc` nor the opportunistic variant of `call/cc` use to
implement continuation attachments.
Reducing external-event polling exposed a problem with fd semaphore
checking, where the check before sleeping didn't abandon the sleep if
an fd semaphore was posted.
Also, fix a bug with interrupted network address lookup.
Improvements include:
- less scattered handling of constant templates
- better recognition of constant templates, like (1 (... 2)), where
the template syntax is not identical to its value
- better code for (x ... ...), where x is trusted
When reading from an input fd blocks, instead of creating a general
event that creates a semaphore, use the semaphore directly (when
available). Also, treat a semaphore internally as an event that
always produces 0.
This change speeds up the "echo" shootout benchmark.
This change speeds up the "echo" shootout benchmark.
When the schemify pass cannot determine that a call is to a primitive
procedure, it generates an `#%app` form that expands to
((extract-procedure rator) rand ...)
Force `extract-procedure` to be inlined (by making it a macro), so the
expansion is
(let ([tmp rator])
((if (#%procedure? tmp) tmp (slow-extract-procedure tmp)) rand ...))
which is usefully faster in the common case that `rator` turns out to
be a primitive procedure.
Chez Scheme doesn't provide `eq-hash-code`, so it's implemented with a
weak `eq?`-based hash table that maps values to fixnums (except for
numbers, symbols, and characters). The table had a lock to support
concurrent use in multiple places, and that became a major source of
contention in parallel builds. Change the implementation to use a new
`eq-hashtable-try-atomic-cell` operation, which effectively moves
contention from the hash table to individual buckets (where it should
be much rarer).
Commit fe708871bd broke cross-module inlining for modules that are
compiled in different Racket processes. The problem is that
cross-module information is represented by prefab structures, and the
change caused Chez Scheme's fasl for prefabs to generate a different
structure type on different runs.
To solve the problem, use `racket/fasl` for cross-module information,
instead. But cross-module information also has inlining information as
correlated objects, so make those supported by `racket/fasl`, too.
The first time a struct is provided through `(contract-out (struct id ....))`,
save `id` to access its transformer binding later.
On reprovides:
- hang on to the original `id`
- use its transformer to recover the original predicate/accessor/mutator names
Also, fix a bug where the order of the mutator ids reported by the
struct info was getting reversed
Probably, nobody noticed that bug. They'd have to work around the renaming
issue in #2572 first.
The default module name resolver uses a cache to map module names to
resolved-path information. The cache was weak in a way that turns out
to be much weaker on Racket CS, essentially because Chez Scheme is
tuned to fire a minor GC more frequently.
The new cache cuts 45 minutes(!) from a 2h15m single-process
distribution build of Racket CS on Linux. That brings it under a factor
of 1.5 of the non-CS build time, instead of over a factor of 2.
Thanks to Caner and Sam for pointing out LONG ago (maybe a year ago)
that the cache works badly for Pycket. Since the cache doesn't make a
big difference for `racketcs -cl racket`, though, it took me this long
to understand that it can be such a big deal for Racket CS when
performing a distribution build.
Help avoid problems with serialization by making the generation of
embedded module symbolic names deterministic and relatively
insensitive to module order. The generated name is based on a
combination of `path->module-path` and paths relative to the
main module of the executable.
Related to #2693
The total time of module name resolver calls is more useful, because
each one takes longer, there should be many fewer, and there are
tasks that end up resolving module paths.
When support for machine-independent bytecode was added, the bootstrap
implementation of linklets ended up being slightly uncooperative.
Source terms from the bootstrap became wrapped as machine-independent
form. For various reasons, things worked anyway, except that
`--linklets` mode prints bytecode instead of S-expressions. Fix the
bootstrap implementation to cooperate correctly.
Related to #2688
Don't discard expressions that will fail due to trying to make a
prefab struct type from a parent that isn't a prefab. Similarly, don't
discard a `make-struct-type` with a built-in property that has a
guard. Don't discard a `make-struct-type-property` with a literal
guard procedure that has the wrong arity.
Related to #2685
Adjust the makefile that ends up in <builddir> with `--enable-cs` or
`--enable-csdefault` (as opposed to `--enable-csonly`) so that `make
racketcs` doesn't imply `make racket3m` if `--enable-racket` has
supplied an existing Racket.
Also fix `make install-cs` related to GRacket for the case that Racket
3m/CGC isn't built.
Relevant to #2683
While a continuation is set up to avoid retaining runstacks, partly by
storing a prompt ID instead of a prompt record, prompt records can
remain on the C stack and get captured anyway. Mitigate that problem
by making the runstack link weak in some prompt record.
Racket CS doesn't have this problem, of course.
Relevant to jeapostrophe/lux#10
Add `single-flonum-available?` and `read-single-flonum`, where the
latter controls whether numbers that have an "s" or "f" exponent
marker are parsed as single-flonums are normal flonums. The parameter
is disabled by default, which changes the meaning of most existing
code that has a literal number with "s" or "f", including `+inf.f`,
`inf.f`, and `+nan.f`.
The compiler constant-folds `single-flonum-available?` and
`real->single-flonum` on a literal number, so use a combination of
those to replace most uses of a single-flonum literal. Single-flonums
within quoted data are less convenient.
After the change that makes printing of struct ignore print-pair-curly-braces
it is possible to simplify the code in io/print and the associated tests.
For traditional Racket, fix `bytes-utf-8-index` to accept 5 arguments
as documented. For Racket CS, fix `bytes-utf-8-index` to return an
index relative to the byte string's start.
Closes#2670
When `print-pair-curly-braces` is true, change the built-in printer to
not use curly braces to group a constructor with its argument.
Restrict its effect to quoted lists, which is more what you expect and
more consistent with `pretty-print`.
Also, change `pretty-print` to not use `{` when using the `list`,
`list*`, `cons`, or `mcons` constructors.
Closes#2662
On case-sensitive filesystems on macOS, these are
distinct (leads to a file not found error). On
case-insensitive systems, the change should not matter.
These changes are intended to address "input port is closed" errors
that have been showing up with Racket CS, possibly because its
scheduler exposed missing synchronization.
Instead of having schemify generate `letrec*` and convert as needed
through a Chez Scheme macro, have schemify perform any necessary
conversion to get the right use-before-definition error messages and
`call/cc` interaction.
This change improves the conversion, since schemify has more
information about bindings, but it also avoids sending Racket terms
through a macro-generating macro at the Chez Scheme level. Avoiding
the macro-generating macro avoids a kind of leak in Chez Scheme, where
a gensym used in a template may become ineligible for GC due to the
way `free-id=?` may both reify the gensym's unique name and attach a
property to the gensym.
Using a will executor to turn a reference from weak to strong still
seems like an ok idea, but it needs to be a regular will executor,
because a custodian-registered value is likely to involve have a
nested self-reference.
More generally, repair the internal `exe-relative-path->complete-path`
function to work when the current directory is not the original
current directory and `racket` is started with a relative path.
Currently, it happens that `exe-relative-path->complete-path` is
called with a potentially different directory only by
`get-lib-search-dirs`.
When the compression format changed to LZ4, which is much faster to
decompress than zlib, the configure script changed to enable
compression by default. Bytecode tends to benefit all around from
compression, but the boot files take 20ms or so longer to load --- not
a lot of time when loading typical amounts of code, but a signficiant
cost for a minimal startup. This commit allows compression to be
controlled separately for boot files, and it configures them as
uncompressed by default.
* Remove irrelevant #ifdefs MZ_USE_JIT
Bonus points - fixes a compiler warning on aarch64 and a typo.
* Fixes a compiler warning on aarch64 for unused current_linklet_native_lambdas
* Simplify conditionals after removing dead store of has_space
The conditional simplification looks good to me. The biggest issue
here was to understand if when `pipe_quote` is true, we can and should
go to the else clause. Actually the more I look at it the more I think
this uncovers and earlier bug where if pipe_quote is true, result and
total_length are left at NULL and 0 respectively after the block.
Change `datum->syntax` so that it limits the transfer of a code
inspector from a source syntax object; the code inspector is kept only
if a macro is being expanded and the macro has the same code inspector
(or, more generally, the weaker of the two code inspectors is
preserved).
This change is a kind of defense-in-depth to prevent the use of
unarmed syntax with `datum->syntax` to access unexported bindings from
the module where a syntax object originates.
The general approach is Ryan's idea. This particular implementation is
a simplification of the general idea, and we'll see whether it's
worakble and sufficient.
The changes in aab63ad3 introduced a dependency on
racket/private/promise, which the analysis was not capable of dropping
due to the use of the `prop:force` property. This caused trouble for the
thread layer, since it introduced a reference to `error`, which is
defined in the io layer. This change adds some additional detection for
struct type properties with guards that accept procedures of particular
arities, which allows `prop:force` to be marked as pure.
Also, a typo in the thread layer’s Makefile meant globals weren’t
actually getting tracked, so this fixes that, too.
`for/fold` is a left fold, which is normally what you want in a
call-by-value language such as Racket, but it makes efficient lazy
iteration difficult. This commit adds a new `for/foldr` iteration form
(along with `for*/` and `/derived` variants) that provides a right fold
operation that offers complete control over precisely how lazy the
iteration ought to be.
In simple microbenchmarks, reimplementing `for/stream` to use
`for/foldr` instead of `for` plus a generator can be almost 40x faster
on large streams.
When `read/recursive` is used, do not inherit parameter values
recorded by an enclosing `read`, and instead look them up again.
This change restores behavior of the old reader.
Closes#2661
When ">" appears in a procedure name, or when other characters appear
that would normally need to be escaped in a symbol, don't add escapes
since `#<....>` isn't readable anyway. This change makes renamed
procedures print in a consistent way with primitive procedures.
Similarly adjust the printing of structure type names.
Closes#2646
Closes#2659 by both recognizing `lib64` as a default path and by
having `--enable-origtree` override inference and specified when
running `configure` through the root makefile.
Swapping the blame before adding #:important context associates the
important party with the negative party for the purposes of picking
“contract violation” versus “broke its own contract” messages in error
reporting. Therefore, only swap after adding the context.
fixes#2531
Instead of limiting the nursery size and performing a full GC every
time a small nursery is full, allow the nursery to be proportional
to the total heap size if generational GC is disabled.
This option allows the user to enable or disable (with
--disable-generations or --enable-generations=no) generations in
3m. Disabling generational collection is, in most cases, a bad
idea, but it may be necessary on a platform where signal handling
doesn't work well enough to support a write barrier that is
implemented with page protection.
Ignore new autoconf variable added in 2.70.
The interesting thing is that debian decided to backport this variable
to their 2.69 release so in some 2.69 autoconf this variable does not
exist but in debian ports 2.69 generates this variable. It is
nonetheless not useful for Racket, so add to ignore list.
When using a built-for-bootstrapping Racket to build Racket CS, the
intermediate module loading module mode should be `--boot` instead of
`--chain`. The repo's top-level makefile takes care of that already,
but not `configure`-generated makefiles as may happen in a build from
a source distribution.
Allows an inaccessible custodian to be GCed, promoting any values that
it manages to its parent custodian. Also repair memory accounting for
custodian boxes.
For values referenced by a custodian, the nature of the custodian's
weak references is slightly different on Racket CS. The reference is
weak enough that the value can be finalized via will (e.g., to close
an unused port), but it's not weak enough to allow weak boxes, weak
hash table keys, or ephemeron keys to be cleared. That's a consequence
of using ordered finalization instead of finalization/weakness levels.
This difference could be avoided at the cost of an extra wrapper for
any finalized value and a discipline of using such wrappers as the
user-visible reference for all custodian-managed values, but semi-weak
references so far appear to be practical and a better compromise.
The use of a will executor for a custodian is a bit of a hack, and it
doesn't want the "keep live until executed" constraint. So, add an
optional internally.
If a late will executor has pending will, then it needs to stay
live until the enclosing place has terminated (and post-custodian
callbacks are run). Otherwise, `ffi/unsafe/alloc` can lose values
that it expects to finalize, and it reports an "internal error".
The late will executor for `register-finalizer` from `ffi/unsafe`
was kept live in traditional Racket, but only as an accident of
custodian shutdown in a terminating place: the shutdown process skips
threads, since that work is technically not necessary. Relying on that
coincidence is asking for trouble, though, so implement retention more
deliberately.
Recognize `(ptr-ref <ptr> _uint8)`, etc., and turn it into a more
direct `(ptr-ref/uint8 <ptr>)`, etc. This improvement speeds PNG
loading by a factor of 10 to 20, for example, because the
implementation expects the pattern to be recognized.
When the number of places approaches the number of available
processing cores, then a spin lock isn't good enough for a small
number of contended hash tables (maybe just one of them). When
contention is discovered, fall back to a mutex-based lock.
When spawning a new subprocess, it's possible that one or more of the
new process's standard input, output, or error descriptors use file
descriptor 0, 1, or 2, even if they don't correspond to any of the
parent process's original standard input, output, or error descriptors.
This can happen if the parent process closes one of its standard
descriptors, and the operating system reuses the file descriptor number
for a new descriptor.
Therefore, be more careful about closing and copying file descriptors in
the child process before calling `exec`. Specifically, move file
descriptors out of the way as needed so they aren't clobbered, and
accommodate cases where multiple standard streams may share the same
file descriptor in the parent process.
fixes#2634
A subcustodian was incorrectly registered as weak for its parent,
which means that an unreferenced custodian could get lost when
shutting down an ancestor.
Register the port, not the file descriptor, especially since a TCP
connection can have ports that share a file descriptor. Also, I think
a weak reference in the custodian doesn't work as intended (visible
through finalization) if the file descriptor is referenced with a
callback that closes over the port.
No `syntax-protect` is needed for `define/private`, etc., because no
new identifiers or expressions are introduced. Adding extra dye packs
can interfere with other macros that pull apart syntax (although maybe
the macros shouldn't do that without using `syntax-disarm`).
A custodian doesn't provide any order on shutting down the objects
that it manages (I was confused about some past experiments), so
avoid that assumption.
Getting the current CPU time is relatively expensive, so get it only
on thread swaps where a thread used its full quantum or 1/100 swaps
otherwise. This approximation should work because thread-specific CPU
times are rarely requested, and they make the most sense for threads
that don't constantly swap out due to synchronization.
Formerly, an expression like `(arity-at-least-value 7)` could crash,
because the `arity-at-least-value` accessor is created in unsafe mode,
and the slow path to accessor errors attempted to use the accessor to
provoke an error message. Instead of using a potentially unsafe
accessor, have the slow path raise an error explicitly with
`raise-argument-arror`. That change has the added benefit of making
error messages mach traditional Racket (at least for structure types
that are not declared as "authentic").
The problem was exposed by tests added in 55dcdf5538.
Instead of a separate hash table mapping continuations to
linklet-instance names, use a continuation mark. That's faster,
because capturing a continuation means copying part of it on continue.
Currently, Check Syntax has trouble correlating `require` forms and
references to imports that go through a macro-introduced rename
transformer. For example, there's no binding arrow from the final
`starting` to the `racket/list` in
#lang racket/base
(require (for-syntax racket/base))
(define-syntax-rule (define-as-first mod starting)
(begin
(require (only-in mod
[first initial]))
(define-syntax starting (make-rename-transformer #'initial))
starting))
(define-as-first racket/list starting)
starting
But change the last two `starting`s to `initial`, and the binding
arrows work.
Until a general repair is in place for Check Syntax, this commit
adjusts 38d612dba6 to use the original export name for an immediate
binding, which acts as a hint to the current Check Syntax
implemenration.
Note that the source-distribution client must have a
"build/ChezScheme" checkout created, maybe by building as a 'cs
variant. A pruned version of that checkout is then included with other
sources. The resulting source distributon then works for building
either Racket variant.
Adapt the configure scripts and makefiles to use a "ChezScheme"
directory that is bundled with sources.
Some expressions like (date-day) gave usually an arity error, but when they
were inlined by the JIT the arity check was wrong, so they produce a segfault
or a nonsensical result.
Provide a way to build Chez Scheme from source using Racket. In the
short run, this lets us distribute source that ultimately depends only
on a C compiler (since a variant of Racket can be built from source
using just a C compiler).
- change an 'an' to 'a'
- remove 'immutable' where expecting either mutable or immutable (don't
bother to specify which, because `vector-common.rkt` doesn't bother)
- remove extra ','
The `poll` system call doesn't work right for fifos, so switch
back to `select`, but use a new strategy to size fd_set buffers
instead of trying to use `getdtablesize` (because the result
of `getdtablesize` can change dynamically on Mac OS).
Also, add a check for input at the rktio level when trying to read
from devices other than regular files. Otherwise, Racket CS (which
doesn't have some redundant polling that is in traditional Racket)
sees spurious EOFs for unconnected fifos.
Closes#2577
* Remove value store in ready_pos but unread
* Move declaration of ready_pos to where it is used
* Make discard of return value of tcp_check_accept explicit
* Split declaration and var assignment to comply with xform
Making `equal?` do the right thing on classes turned out to be easy---it
just involved adding a straightforward `prop:equal+hash` property to the
`class` struct—but making it work properly for *objects* was the tricky
part. The trouble is that `equal?` on objects that don’t implement the
`equal<%>` interface is just ordinary structure equality, which can be
relevant if objects are inspectable. Writing `(inspect #f)` in a class
body is like making a struct `#:transparent`, and it has all the same
ramifications for equality.
The trouble is that `class/c` creates new wrapper classes, and every
class has its own struct type. Since the default behavior of `equal?` on
structs is to *never* be equal to structs of different types, even
subtypes, an object created from a contracted class can never be
`equal?` to an object created from the same class without contracts.
The solution is to add a `prop:equal+hash` property to `object%` itself
that emulates the default behavior of `equal?`, but sees through class
contract wrappers. Since struct type properties are inherited by
subtypes, this property will be present on all objects, and it only
needs to be attached once.
fixes#2279
Mainly, this improves `make-keyword-procedure`: when applied to a single
argument, it now uses `procedure-rename` to ensure the resulting
procedure has the appropriate name. A couple other changes also guard
against the case where a lambda expression has no inferred name and no
source locations information, which would lead to the source locations
in the implementation being used, instead.
