If the sub-template inside #(...) is unsyntax-splicing instead
of list, produce the template #((~@! . ????)) instead of calling
(datum->syntax o list->vector o syntax->list). Fixes#2402.
Fix some race conditions involving concurrent setup tasks that are
each trying to generate both machine-independent bytecode and
machine-specific bytecode.
add a function to escape any glob wildcards in a path or string
also add a private `glob-element->filename` function so that, e.g., the pattern
`a\*` matches the file named `a*` (previously, the match would fail and
I think it was impossible to match for only `a*`)
Fix the fallback interpreter (which is used for the "outside" of a
module that is too big to compile) so that it's safe-for-space.
This change is unlikely to repair any immediate problems, but space
safety problems are difficult to detect and avoid when the underling
implementation is not safe-for-space so fixing the interpreter is
likely worthwhie in the long run.
Module definitions and expression need to have a prompt around them to
delimit continuation capture, variable assignment needs to happen at
the right point to ensure that reassignment is guarded and
non-assignment is detected. But avoid the prompt when it's not needed,
such as around function definitions.
Closes#2398
Similar to a255def019, but for side effects potentially
exposed by definition RHS expressions, instead of
expressions not in a definition. Improve that commit and
this one by only forcing variable assignments at non-simple
expressions.
Travis is eliminating its container-based infrastructure
and deprecating the `sudo` keyword.
This commit also updates the example build matrix to use
more recent Racket versions.
Corresponds to https://github.com/greghendershott/travis-racket/pull/29
Discard local-variable names to avoid `gensym` artifacts in the same
way that a more complete compilation would discard the names. This
change does not affect function names, which are preserved through
separate properties.
It most cases, it's more important for `compiler/cm` to reliably
replace a file that might be busy than to make the file update atomic.
To suport that kind of use, `call-with-atomic-output-file` implemented
a fairly reliable, multi-step, non-atomic process for replacing a file
on Windows.
For recompilation of bytecode in machine-independent form, however,
`compiler/cm` now really wants to atomically write a replacement
bytecode file. That's not generally possible on Windows (except on
NTFS with transactions, which are discouraged...), but MoveFileEx work
atomically in some cases and it's likely to work for the cases needed
by `compiler/cm`. Probably.
So, add a mode to `call-with-atomic-output-file` to get "more atomic"
updates on Windows. This mode is enabled by a callback that makes the
caller responsible for deciding what to do with the move fails, such
as waiting a while and trying again. And `compiler/cm` now waits a
while and tries again, up to a limit, which should be good enough for
recompilation.
Enable `raco {setup|make}` to build two sets of compiled files: one
set that is suitable for the current machine, and another set that is
suitable for a different machine or for all machines (i.e.,
machine-independent bytecode).
In the long run, this new `raco setup` mode support cross compilation
where the build machine and target machine have different bytecode
formats --- unlike the current cross-compliation mode, which relies on
there being a single bytecode format in traditional Racket for all
platforms.
In the short run, the new mode enables the faster creation of
Racket-on-Chez distribution builds. The build server can send out
machine-independent bytecode to client machines while using
machine-specific bytecode for itself to drive the build process.
The new compilation mode relies on a somewhat delicate balance of the
`current-compile-target-machine` and `current-compiled-file-roots`
parameters (as reflected by the `-M` and `-R` command-line flags for
Racket) as well as cross-compilation mode (as enabled by the `-C`
command-line flag).
The 'target-machine result from `system-type` reports the
default value of `current-compile-target-machine`.
Also, fill in pieces to make `setup/cross-system` work
for RacketCS, although cross-compilation is still several
steps away.
The new path for recompiling from machine-independent files
trues to read a ".zo" file without holding the recmopilation
lock and without an `exn:fail:filesystem` handler.
Wait until replacement is more assured before deleting an existing
".zo" file.
Also, don't delete a ".zo" file that is later in the
`current-compiled-file-roots` search path than the one being written.
This refinement supports setting up a search path to try
machine-specific compiled files and fall back to machine-independent
files, for example.
Add `-M`/`--compile-any` to `raco setup`, `raco pkg install`, etc., to
build machine-independent bytecode, which is useful in the process of
building distributions.
The `parallel-lock-client` protocol expects a #f back when a
file was meanwhile compiled by another process. So, don't
just forget about a file after it is compiled, in case there
is still a lock request on the way for that file.
Actually, the machine-independent-to-specific part is trivial. The
hard part was making `compiled-expression-recompile` enable
cross-linklet optimization as it recompiles, since that involves
pulling apart metadata and putting it back together afterward.
The `compile-machine-indendent` parameter controls whether `compile`
creates a compiled expression that writes (usually in a ".zo" file) to
a machine-independent form that works for anhy Racket platform and
virtual machine. The parameter can be set through the
`-M`/`--compile-any` command-line flag or the `PLT_COMPILE_ANY`
environment variable.
Loading machine-independent code is too slow for many purposes, but
separating macro expansion from backend compilation seems likely to be
a piece of the puzzle from cross-compilation and faster distribution
builds.
Converting "invalid memory reference" to an `exn:fail:contract` (which
is the default conversion) hides crashes as success when a test
expects an error.
Also, fix a bug that was hiding as an expected excdeption.
The Racket and RacketCS implementations had separate copies of
linklet-directory and linklet-bundle reading and writing. Move the
implementation into the expander layer.
The primitive '#%linklet instance now omits directory and bundle
operations and `read-compiled-linklet`. It intead must provide
`write-linklet-bundle-hash`, `read-linklet-bundle-hash`, and
`linklet-virtual-machine-bytes`.
In particular, when there isn't any redundancy detected, then
just make a single call into the projection and create just a single
class.
This seems to help on at least one of the configurations of
dungeon, which completes in about 6 minutes with this commit
and I gave up waiting after 15 minutes for the version of
racket that didn't have it
Improves the error message for:
```
(define-syntax (like-lambda stx)
(syntax-case stx ()
[(_ e) #'(lambda () e)]))
(like-lambda (define x 1))
```
Based on a report from @pkoronkevich.
When an executable distibution is created, some path become
unavailable at run time, such as the result of `find-links-file`.
Change the contract on those functions and adjust the implementation
to return `#f` in those cases. This is a backward-compatible change in
the sense that uses that now return `#f` would have crashed before
(although it does shift the blame in that case).
Based on an initial patch by Shu-Hung.
Closes#2352
Source mode was a leftover from early iterations of the expander. A
bootstrapping mode that uses replacement `compile-linklet`, etc.,
turned out better.
For consistency with traditional Racket and currently matters on
Windows. The Windows implementation of file-truncate should probably
not move the file position as it does, though.
In GUI-application mode (e.g., running GRacket), a console is allocated
on demand if a program tries to use the original ports. Move that
on-demand handling into rktio, where it's simpler and works for
RacketCS.
One more take on the problem addressed by 990e1f1e30. This adjustment
avoids copying properties from the original form to the identifier
that is preserved in 'origin.
The `get-compiled-file-sha1` function assumed that a ".dep" file is
up-to-date when present. That may not be consistent with all uses,
including in `file-stamp-in-paths` as used by DrRacket for "populate
compiled", and an old file can go wrong with the recent ".dep" format
change. Make `get-compiled-file-sha1` at least check the version on
the ".dep" content before trying to use it.
Relevant to #2354
A function that uses `call-with-immediate-continuation-mark` in tail
position should not be flagged as "preserves marks", because the JIT
needs to bump the mark stack if the function is called in non-tail
position.
Closes#2333
The repair is more precisely a repait to xform, which incorrectly
parsed a C function definition that starts "struct" as a struct
declaration. (The function starts "struct" because the return type is
"struct Scheme_Overflow_Jmp *".) Since the function wasn't recognized,
xform didn't convert it to cooperate with the garbage collector.
Closes#2341
Previously, the following program would print "error writing to
stream port" on program exit.
(define cust (make-custodian))
(define out
(parameterize ((current-custodian cust))
(open-output-file "test.data" #:exists 'truncate)))
(write-string "This needs flushing...\n" out)
(custodian-shutdown-all cust)
(exit 0)
So far, bytecode for traditional Racket has been kept separate from
RacketCS bytecode by using a different "compiled" subdirectory for
RacketCS. That makes sense for development work to allow the
implementations to coexist, but it creates trouble for packaging and
distributions, and it (hopefully) won't seem necessary in the long
run. Treating the different virtual machines like different versions
seems more generally in line with our current infrastructure.
Rearrange the configure scripts so that it will be possible to build
RacketCS from a source distribution and have it installed in the right
place. Also, when building Racket3m just to bootstrap RacketCS, don't
install Racket3m.
Retains a strong link to a place-channel write end when there's at
least one waiting thread. This is symmetic to keeping a strong link to
the read end when the place-channel queue is non-empty. The change
repairs a problem building documentation with places in `racocs
setup`.
Refines 2ef8d60cc6 to avoid characterizing the failure as a `(-> any)`
contract on `hash-ref`, since `hash-ref` doesn't enforce that contract
in general. Go back to an `exn:fail:contract:arity` error, but keep
the specialization of the error message to clarify that it's from
`hash-ref`. Also, bring RacketCS into sync.
Although a `directory-exists?` check is useful for providing better
error messages, it's fundentally a race condition, since an external
process can always remove a directory between the check and a use of
the directory. Because of that limitation of `directory-exists?`, we
normally avoid making it part of a contract. This commit adjust
937aa3cdb1 to follow that convention while preserving the helpful
check and documentation improvements.
Their semantics assume all directory `path-string?` arguments point
to existing directories in the filesystem but they do not actually
check to verify resulting in unhelpful inner exceptions
breaking the functions' semantic abstractions.
Fixed by adding appropriate checks.
Test cases included too.
Documentation updated to reflect the requirement for paths to
refer to existing directories.
Also added note that `generate-stripped-directory` does not
compile or render source files.
When catalog is specified via file:// URL to a local directory with local
package directories that have been stripped in various modes, `raco pkg install`
will incorrectly error out with an incompatible package content error when
a binary strip mode is selected and a file-system error when source strip
mode is selected.
The cause is due to the file:// URL's resolved file path not being passed
along to the helper functions in the various code paths handling the different
strip modes; instead the full original file:// URL is passed along and is
misinterpreted as an actual filesystem path resulting in the observed failure
modes.
The repair is simple; change the relevant argument so the resolved filesystem path
is used instead of the original file:// URL.
Test cases added to test various combinations of strip modes when using
`raco pkg install` and local catalog directory.
When a file descriptor cannot be `dup`ed for a place message, the
error message has to be delayed until the partially copied message is
cleaned up. Various problems with the message-serialization code
caused that delay to work incorrectly or incompletely.
Closes#2305
The repair in 4396b841c0 for internal names exposed a problem with the
way `linklet` handled renaming on export, where it mixed up the names
that should be used internally and externally in errors.
Merge to v7.1
Lots of plumbling was in place to preserve the source name (instead of
the symbol generated to avoid collisions for macro-introduced
definitions), but some small pieces were missing.
Closes#2288
When upgrading native-library versions, a still-relevant
patch got lost. The patch corrects a problem with empty
glyphs getting treated as missing glyphs.
Closesracket/pict#42
The pattern
(ephemeron-value
(hash-ref! intern key
(lambda ()
(make-ephemeron key (wrap key)))))
is wrong, because a GC might happen between the time that the
epehemeron is found in the table (or the time that the key was just
added to the table) and the time that `ephemeron-value` is called to
extract the value. If the key is not otherwise accessible, the value
may no longer be in the ephemeron.
If `make cs` is run without specifying a SCHEME_SRC, then make sure
that `configure` and `make` are re-run in the Chez Scheme checkout,
in case it was updated.
A collection can only invoke certain callabcks (e.g., for DrRacket's
GC icon) when the collection is performed in the main thread. Also,
delay posting GC logging events to receivers that cannot work at
interrupt time.
This reverts commit 41fd4f3a5e.
The problems this change was intended to solve can be solved in other
ways, without loosening guarantees about expansion order. See the
discussion in #2154 for more details.
closes#2154
The procsses-based build was technically broken with the addition of a
"prefetch" thread (some time back) to improve parallelism, because the
`write`-based implementation of messages did not protect again
interleaving by different threads. The problem turns out to be easier
to expose when running with RacketCS.
Formerly, `--scope-dir` would include only the specified directory in
the search path for already installed packages, etc., which means that
it would only work right as a kind of installation scope that is a
step beyond "installation" on the "user"-to-"installation" spectrum.
The `'pkgs-search-dirs` confiugration entry, meanwhile, provides more
control over search ordering in installation scope. Make `--scope-dir`
work more consistently with that search-path configration.
This change also makes "instllation"-scope operations use the search
path more consistently, since some actions used to use the whole
search list while others pruned any prefix before the main
installation directory in the search list.
Add a way to declare an Objective-C method call as blocking in the
sense of the `#:blocking?` argument to `_cprocedure`.
Usingf `with-blocking-tell` should allow the Cocoa backend for
`racket/gui` to wait for events in the main place without blocking
other places.
Fix various problems with the implementation of places, and let
`processor-count` return the actual number of processors. A parallel
build via `raco setup` seems to work but not scale well.
Instead of defining `rktio_fd_t` to be independent of a `rktio_t`,
which isnt quite true, introduce `rktio_fd_detach` and
`rktio_fd_attach` functions to make a transfer explicit, such as when
a file descriptor goes through a place channel.
This adjustment avoids a corner case in cleaning up a file descriptor
from an abandoned channel, where the finalizer might run in a Chez
Scheme thread that is not associated with any place.
When a single-use function is inlined late enough in the optimization
process, and when the body has the only use of some variable computed
by an expression that can be move in place of the use in the inlined
function (but not in the non-inlined function), then it's a problem if
the function binding isn't pruned away early enough. Make sure the
binding to the function is marked as unused after the function is
inlined.
This bug was exposed by a recent change to the "dssl2" package.
Implement place channels and messages, and change `place-enabled?` to
claim that places are enabled, but `processor-count` still reports 1.
The implementation of place channels has an interesting use of
ephemerons --- that is, a use that isn't just solving a key-in-value
problem. Using epehemerons solves the problem of forgetting a place
channel and any thread blocked on the read end when there are no
producers on the write end. Along similar lines, when only the write
end is retained (i.e., no readers), the channel data is forgotten and
writes become a no-op. The read end holds a "read key" and references
the channel data through an ephemeron keyed by a "write key"; the
write end similarly holds a "write key" and uses an ephemeron keyed by
the "read key". This use of an epehemeron implements a reachability
"and": retain the place-channel data only if the read end *and* the
write end are both reachable. (Minor point: a read end also holds onto
the "write key" anytime the channel already has data.)
The Rumble layer provides a primitive `fork-place` operation and
`start-place` hook to tie in place initialization from other layers
(especially "io"), but the rest can live in the "thread" layer,
especially to handle place-channel synchronization.
When a stand-alone executable created by `raco exe` needs to load
modules that start with a `#lang` line, there have been various
obstacles to adding the right run-time support via `++lib`. The
`++lang` flag addresses those problems and makes it easy to indicate
that enough should be embedded to support loading modules with a
specified language.
There are problems in the way that various handlers interact for the
"lang/reader.rkt" versus `(submod "." reader)` search path that
converts a language name to a reader. To accomodate the search in a
standalone executable (that does not provide access to collections in
general), the module name resolver must refrain from raising an
exception for a non-existent submodule path that refers to a
non-existent collection.
There's no place-channel communication yet --- just enough of a
conversion to thread-load storage to make places possible.
In contrast to traditional Racket, where the expander linklet is
instantiated once per place, the flattened expander linklet is
instantiated only once in RacketCS (because it's inlined into a Chez
Scheme library). The expander therefore needs to keep per-place state
separate, and the same for the thread, io, and regexp laters.
In the expander/thread/io/regexp source, place-local state is put in
an unsafe place-local cell. For traditional Racket, a place-local cell
is just a box. For RacketCS, the thread through expander layers are
compiled in a way that maps each cell to a fixed index in a vector
that is stored in a virtual register, so the value is roughly two
pointer indirections away (thread context -> virtual register array ->
place-local vector). Multiple Chez Scheme threads in a place, such as
threads to run futures, share the same place-local vector.
Although `place-enabled?` reports #f, `dynamic-place` from `'#%place`
can create a place as a Chez Scheme thread and load a module there.
An extra scope is needed to separate the bindings of a `letrec` from
the `letrec` body, in case a macro moves right-hand-side expressions
to the body.
Michael Ballantyne and William Hatch reported this problem and its
solution in December 2016, but I forgot to add the repair.
Relevant to #2237
The repair in 7176fc4253 did not make the no-discard decision stick
well enough for some cases. Robby found this bug using the Redex model
and random testing, too.
Using a parameter for the current expansion context means that if a
macro spawns a thread, the thread thinks that it's in an expansion
context. Switching to a raw continuation mark avoids that problem.
Along the way, bring Racket and RacketCS more in line by making both
have an internal notion of "root" prompt tag that can be used to get
all continuation marks independent of any prompts. That's not structly
necessary, since a continuation mark could be combined with a distinct
tag to make the mark always accessible, but it's simpler and more
lightweight to use a root prompt tag.
This improvement removes only a few places where a variable use is
considered possibly too early in the RacketCS implementation, but
the improvement is significant for uses of `input-port?` in "io".
Allow a single argument to comparison functions like `<`, and
support the same arities as the generic version for fixnum and
flonum operations like `fx+` or `fl+`.
RacketCS weak `equal?`-based hash tables didn't retain flonums, and
hashing of hash tables was not properly insensitive to the order of
keys.
Racket, meanwhile, didn't limit work consistently for different kinds
of hash tables, and it didn't keep a counter value odd as intended
(but the counter never gets large enough to appear to be a mapped
pointer, anyway).
Racket did not check for a break when escaping from a break-disabled
context to a break-enabled context. RacketCS didn't check in other
cases, either. Fix those various cases.
Commit 1afcbee381 effctively dropped a conditional case that was
marked as "shouldn't happen" --- but it does happen and makes sense.
Adjust the replacement `delete-directory/files` call to accomodate the
case.
Relevant to #2198 and #2236
Part of this change restores a `++direct` that was lost in 98ae91e0ba
for "racket/src/thread" to make the atomicity state a virtual
register. Also make `display` on a byte string more directly call
`write-bytes`. That change restores a 5-10% speed improvement for
`racketcs -cl racket/base`.
Change the expander-performance macro so that it's a very low cost if
not enabled on startup. An extra JIT specialization reduces the cost
further, since the enabled state is known by JIT time.
When a child place terminates, the parent's count of the child's
memory use needs to be updated. Until this repair, the
`current-memory-use` function has been reporting an incorrectly large
number after a place terminates.
Fix liveness for "simple" arguments to inlined functions. Fix
handling of non-authrntic structure access and mutation to
allow the possibility of a GC.
Also, while we're at it, make the functions produced by curry cooperate
better with other parts of Racket. Namely, make the information reported
by procedure-arity and procedure-keywords accurate, and give procedures
more useful dynamic names.
The mask encoding of an arity is often easier to test and manipulate,
and masked-based functions are sometimes faster than functions that
used the old arity representation (while always being at least as
fast).
Attempting to assign an arity like `(expt 2 100)` to `(lambda x x)`
won't work anymore; it will raise an out-of-memory exception, because
the arity is represented internally as a mask. The arities that cannot
be represented aren't sensible arities, anyway.
The main performance improvement is in calling a function returned by
`procedure-{reduce-arity,rename}` when the arity is not a single
integer. Calls to functions with > 29 arguments can be worse, but
that seems like a much rarer case.
Forcing JIT code generation through an environment variable is useful
to get a sense of how much machine code is generated for a program.
Setting `PLT_LINKLET_TIMES` causes the overall memory used by
JIT-generated code (including adminstrative overhead) to be printed on
exit.
In pass 1, syntax class defns (without #:attributes decl) need to parse
patterns to determine exported attributes. But to allow forward refs,
stxclass references are not resolved until pass 2.
Previously, this was done by just reparsing patterns in pass 2. But that
means pattern expanders get expanded twice, and their expansions might
not agree (eg generate-temporaries). So instead, parse in pass 1, insert
"fixup" patterns to delay stxclass resolution, and resolve fixups in pass 2.
Complication: a fixup pattern is assumed S, but can change to H in pass 2.
The actual problem is that build-chaperone-contract-property
exported to the user defaults #:exercise and #:generate
to false. This commit changes the default fallback value
in the case where #:generate is not a procedure instead
of changing build-chaperone-contract-property directly
to stay consistent with the how contract-struct-exercise
currently does it.
The default values updated in commit ffc5720b5 do
not work for very subtle reasons. In build-contract
in racket/contract/private/prop, the default values
should not accept an extra ctc argument since ctc
is already handled by make-flat-contract. The
default gen procedure should also be (λ (fuel) #f)
instead of (λ (ctc) (λ () #f)) since the latter
would generate false when the generation should
have failed. In build-property, the default procedure
(λ (ctc) (λ (fuel) #f)) is correct and should not
be changed to (λ (ctc) (λ () #f)).
Improve the `hash-ref` error message when the failure result does not
accept zero arguments. (This only changes what the error messages says.)
Example:
```(hash-ref #hash() 'a add1)```
Old message:
```
; add1: arity mismatch;
; the expected number of arguments does not match the given number
; expected: 1
; given: 0
```
New message:
```
; hash-ref: contract violation
; expected: (-> any)
; given: #<procedure:add1>
; argument position: 3rd
```
When a `define` that shadows a `require` appears before the `require`,
then the `require` may fail to other, non-shadowed bindings from the
same `require` spec.
Thanks to Matthias for reporting the problem.
The new argument to `hash-iterate-value` and most other
`...-hash-iterate-...` functions determines a result to be returned in
place of raising a "bad index" exception.
For most kinds of hash tables, a "bad index" exception will only
happen when the provided index is wrong or when a hash table is
mutated during an iteration. Mutation during iteration is generally a
bad idea, but in the case of a weak hash table, a potential background
mutation by the garbage collector is difficult to suppress or ignore.
Adding an option to control bad-index behavior makes it easier to
write loops that defend against uncooperative tables, including loops
where a hash-table key disappears asynchronously.
Racket's printer was already using this functionality internally, so
the change to `hash-iterate-value` and company mostly exposes existing
functionality.
The `in-hash` form and related sequence constructors similarly support
a bad-index alternate value so iterations can handle that case
explicitly. They do not use the new bad-index support implicitly to
skip missing entries, because that idea does not play well with the
iteration API. A hash-table index can go bad after `in-hash` has
selected the index and determined that it should be used for the next
iteration, and a sequence can't take back that decision.
The `ffi/unsafe/collect-callback` library exposes functionality
formerly only available via Racket's C interface, but implement
it for both Racket and RacketCs.
Make `map` inline again, which involves optimizing away
(variable-reference-from-unsafe? (#%variable-reference))
early enough. Fix post-schemify optimization for `procedure?
by adding both forms of an import name to the `imports` table.
Fix a problem with inlining operations passed to an inlined
function (as reflected by the addition of `find-known+import`).
At phase 1 and higher, the expander tentatively allows an unbound
identifier so that, for example, `define-for-syntax` can define a
helper function syntactically after a compile-time expression that
uses the helper. While unbound references eventually trigger an error,
the reordering can be consuing, as in the example
#lang racket
(define-syntax (f stx)
(syntax-parse stx
[(_ oops) #'ok]))
which complains about `_` when the real problem is that `syntax-parse`
isn't imported.
To provide better errors, `raise-syntax-error` now implicitly extends
an error message to include a list of previously encountered unbound
identifiersin the current compilation unit. That list will be
non-empty only at phase >= 1. With that change, the error message for the
above example is
bad.rkt:5:5: _: wildcard not allowed as an expression
after encountering unbound identifier (which is possibly the real problem):
syntax-parse
in: (_ oops)
....
Closes#2167
The expander's output normally uses a distinct symbolic name for every
distinct binding within a linklet. That property is useful for
consumers like schemify, but it's counterproductive for minimizing the
diff in changes to "startup.inc", since the traditional Racket
compiler doesn't need that guarantee. Use `--local-rename` to generate
"startup.inc", which should make future diffs smaller and more
composable after changes to the expander.
* Fix fasl bug in Racket 7.0 beta
The following program causes racket to error in Racket 7.
(parameterize ([current-write-relative-directory (build-path "/" "a")])
(s-exp->fasl (build-path "/")))
This bug appears to have been introduced in Racket 7, and not in
Racket 6.x.
* Fix another bug where 'same was put through path-element->bytes
* "/" => (car (root-paths-list))
This is for windows where simply "/" is not a complete path.
* Add similar tests to serialize library.
* Better error message when relative-directory is a bad pair
Before it would give an internal list-tail error, now it returns
a proper bad argument error.
* Better tests, and improved common case
Now that `ffi/unsafe/alloc` deallocations are triggered by a place
exit, it's more likely that an ffi-call lock can be contended during a
place's termination. When that happens, the place cannot cooperate as
usual. Accomodate this rare situation by spinning.
The repair in 49a90ba75e reorders two lines in a way that, in
retrospect, seems worrying. I can't construct an example that goes
wrong, so maybe it's fine, but it seems possible (now or with some
future change) that attempting to visit available modules could lead
to the same attempt in the same thread and therefore a loop.
This commit changes the repair to just always take the lock instead of
fixing up the attempted shortcut. There's a tiny extra cost to always
taking the lock, but that extra cost seems like a better choice
overall.
If interrupts are disabled to prevent thread swaps, then don't react
to `end-atomic` by performing a deferred swap. That might happen
with logger callbacks after a GC where a deferred action was
overlooked due to a rare race condition.
Now that Chez Scheme supports libraries in boot files, make
"racket.so" work when it is converted to a boot file. Loading it as a
boot file can save around 20ms on every major GC, since the
"racket.so" code will be in the static generation.
For now, however, `racketcs` is not set up to use "racket.so" in boot
form.
* Relative paths should still be readable.
The resent PR to enable relative serialization resulted in serialzed
objects that weren't actually readable (containing literal path
elements). This PR converts them to bytes.
* Move from serialize to relative path.
Also change path->bytes to path-element->bytes
* Path elements can also be 'up and 'same.
Also merge in the relevant code from racket/fasl.
Use an association list instead of an eq hashtable. This choice is
compatible with assumptions in traditional Racket (i.e., that the
number of mark keys per continuation frame will be small) and cuts
about 1/4 of the time in a benchmark like
(define f
((contract (-> (-> integer? integer?))
(λ () values)
'pos 'neg)))
(time
(for ([x (in-range 1000000)])
(f x)))
* Add allow the binder in prop:serialize to be a procedure.
This procedure is evaluated at serialize time, and is useful if
the deserializer is not known during object-type creation time,
but is during serialize time.
* Add docs+tests.
* Add a history note.
* Add option to create relative paths for 'serialize'
Serialize would previously always create complete byte-string paths.
This adds an optional parameter to serialize (#:relative-to) to enable
relative path creation.
Now when deserialize finds a relative path, it resolves it with
respect to `current-load-relative-directory`.
* Moved fasl's path<->relative-path-elements functions
I moved it into the private/relative-path module, so that serialize
can make use of it.
* Update serialize to use relative-path.
* Add tests.
* And update docs.
When expanding a `(module* _name #f ...)` submodule, accumulate all
module scopes on the `#f` and use the `#f` for a reexpansion.
Attempting to start each time from the enclosing module's scope loses
scopes that were generated from previous expansions. One consequence
of losind a scope is that an original definition may appear to be
macro-introduced, and the defined variable may become inaccessible
in the module's namespace.
Generating the code for a `_fun` type takes hundreds ot thousands
of times as long as in the traditional Racket VM, to cache results
to reduce the cost.
Further correct the implementation of `variable-reference-constant?`
on bindings to primitive variable.
This repair affects method-call ctype caching in `ffi/unsafe/objc`.
Add some logging there to make problems easier to detect. Also,
add and improve linklet-evel performance logging for comparing
the traditional Racket VM to Racket-on-Chez.
When setting up the namespaces that imitate primitive instances,
the "constant" annotation wasn't set. The v6.x expander gets this
wrong, too, for different reasons.
Make schemify inline structure accessors and mutators across linklet
boundaries --- or, in JIT mode, across function boundaries --- by
replacing an accessor or mutator with a `#%record?` test and
`unsafe-struct*-{ref,set!}` operation.
When linklets are compiled in JIT mode and a called procedure is to be
compiled on demand, consult a cache of compiled fragments (by default,
"jit.sqlite" in the addons directory) and either use an existing
compiled fragment or add to the cache after compiling.
Results for this initial implementation suggests that the idea is
workable. With the cache, starting a JIT-mode program a second time is
almost as fast as non-JIT mode (i.e., directly loading machine code).
Some refinements are needed: limiting the size of the JIT-fragment
cache, better contention handling, and better inlining of structure
operations in JIT mode (which may be useful to cross-linklet
optimization in non-JIT mode, too).
To avoid recording absolute paths from a build environment in bytecode
files, the bytecode writer converts paths to relative form based on
`current-write-relative-directory`. For paths that cannot be made
relative in that way and that are in source locations in syntax
objects, the printer in v6.x converted those paths to strings that
drop most of the path.
The v7 expander serializes syntax objects as part of `compile` instead
of `write`, so it can't truncate paths in the traditional way. To help
out the expander, the core `write` function for compiled code now
allows `srcloc` values --- as long as the source field is a path,
string, byte string, symbol, or #f. (Constraining the source field
avoids various problems, including problems that could be created by
cyclic values.) As the core `write` for compiled code prints a path,
it truncates a source path in the traditional way.
The expander doesn't constrain source locations in syntax objects to
have path, string, etc., source values. It can serialize syntax
objects with non-path source values at `compile` time, so there's no
loss of functionality.
The end result is to fix abolute paths that were getting stored in the
bytecode for compiled packages, since that's no good for installing
packages in built form (which happens, for example, during a
distribution build).
Although SHA-1 hashing functions are available from `openssl`
libraries, a fast crytopgraphic hash is useful for many purposes below
the layer where the OpenSSL library has been opened. And SHA-1 is
reasonably easy to add to rktio.
Meanwhile, provide an equally convenient SHA-2 function to discourage
bad security practices (i.e., using SHA-1 where SHA-2 should be
preferred).
Applying jitify to a linklet now generates fragments of code that
are not nested. The drawback of this approach is that calling
a nested function needs an extra indirection, and the closure
has an extra slot. The advantage is that the fragments can be
separately compiled and fasled, which could enable a cache of
compiled fragments.
Use a `(let ([<name> ....]) <name>)` wrapper to communicate
an 'inferred-name property from correlated objects to
Chez Scheme. This stategy relies on a Chez Scheme patch to
make the wrapper work consistently.
The improvements reported in 74012f8c57 were actually due to a broken
experiment that dropped source locations on application forms, instead
of preserving them in marshaled code. Adjust the expansion pipeline
to do that earlier and intentionally.
The xify pas doesn't help all that much after all, but it's still more
comfortable to be independent of local-variable names.
The xify pass replaces local variable names with `x0`, `x1`, etc.
Using a minimal set of symbols makes the fasled form smaller
and typically take only 60-70% as long to read.
The places test suite included some tests that create lots of places
and don't wait for them, which can lead to an overload of places that
exhausts resources such as file descriptors. Improve the tests, and
also improve a failure behavior from a crash to an error message.
The `place-kill` function sends a message to another place to
terminate, but it didn't wait for that message to take
effect before returning. Worse, it put the place object in a
state that claimed that the place had terminated.
When the first subexpression is complex and the second
is a literal character, the generated JIT code swaps the
argument order, but compilation didn't swap the test for
whether one or the other is a literal character to skip
a run-time test.
When an embedding application calls `scheme_basic_env` a
second time, it's supposed to reset the main namespace, but
the new expander wasn't reset correctly.
* Fix handling of single-percision infinities and nan
* Document that non-`hash-eq?` hash tables are accepted by `jsexpr?`.
* Document that the value of `json-null` is recognized using `eq?`
* Use `case` instead of `assoc`.
* Use contracts
Delay reporting of potential problems until an actual problem
is detected. Correct a mismatch between original and renamed
symbols to restore detection of problems.
* When you delete a file in Windows, then the name doesn't go away
until the file is closed in all processes (and background tasks like
search indexing may open files behind your back). Worse, attempting
to create a new file with the same name reports a permission error,
not a file-exists error; there's seems to be no way to tell whether
a permission error was really a file-exists error.
This creates trouble for `make-temporary-file` when files are
created, deleted, and created again quickly enough and when
something like a search indexer runs in the background (which is the
usual Windows configuration). In practice, that kind of collision
happens often for `raco setup` on my machine.
To compensate, make `make-temporary-file` try up to 32 times on a
permission error. A collision that many times seems extremely
unlikely, and it seems ok to delay an actual permission error.
Windows provides a GetTempFileName function from "kernel.dll" that
must be able to deal with this somehow --- perhaps because it's in
the kernel --- but it doesn't solve the problem for making temporary
directories, hence the 32-tries approach for now.
* When a deleted file's name persists because the file is open in some
process, then a directory containing the file cannot be deleted.
This creates trouble for `delete-directory/files`, since
`delete-file` on a directory's content doesn't necessarily make the
directory empty. In practice, this happens often for package
upgrades on my machine, where the package system wants to delete a
short-lived working space that the indexer is trying to scan.
To compenstate, change `delete-directory/files` to delete a file by
first moving it to the temporary directory with a fresh name, and
then delete the file there. It may take a while for a file to
disappear from the temporary directory, but meanwhile it's not in
the way of the original enclosing directory.
* When a file is open by any process, it prevents renaming any
ancestor directory of the file.
This creates trouble for the package system, which installs a
package by unpacking it in a temporary place and then moving it by
renaming. The package system also removes a package by renaming it
to a subdirectory of a ".trash" directory. If a background indexer
has a package file open, the move fails. In practice, a move fails
often on my machine when I'm attempting to upgrade many packages.
To compensate, make the package system fall back to copy + delete
if moving fails with a permission error.
Bind variables in a way that allows `local-expand` (with an empty stop
list) to replace a reference to the binding with one that has the same
scopes as the binding.
This repair was motivated by tests in the "rex" package. The
new test added here failed before by finding 'new both times,
but in the "rex" case, the mixup led to the same variable
being imported and exported at the linklet level.
struct-out was putting the super-struct's accessors into two parts of a
struct-info: the accessor list and the mutator list
this commit puts the accessors only in the accessor list and the
mutators in the mutator list
Using a frame pointer for the ABI of internal helper functions
should make the stack friendlier to tools like `perf`. There
may be a small performance cost, though.
Alexis's repair, and as she notes, forcing a `post-expansion` context
value in the core `#%module-begin` expander may allow a simplification
in "definition-context.rkt". But it's not immediately obvious, so save
that potential improvement for later.
Relevant to #2118
When `local-expand` receives one or more internal definition contexts,
it would forget about any current post-expansion scopes. That's
particularly a problem in a 'module-begin expansion context, where the
post-expansion scope ensures that any bindings are suitably
phase-specific.
Closes#2115
This is similar to the recent change of functions with optional
values. Using unsafe-undefined instead of a gensym makes it easier
to avoid the check of the missing argument.
When the separator is a string, these function construct a regexp
that is cached to make repeated calls faster. But when the string
is mutated it is necessary to recalculate the regexp.
Commit 32b256886e adds shifts in one place where it shoouldn't;
the "determinsitic-zo" test exposed the problem.
Also, avoid adding shifts that will have no effect, which avoids
accumulating useless shifts in some top-level contexts.
Various parts of the expander, including `local-expand`, always
flipped the use-site scope when flipping an introduction scope. Onlt
`syntax-local-introduce` should flip both of them, though.
Closes#2112
When expanding in a namespace for a module unmarshaled from ".zo"
form, a scope corresponding to the module's "inside edge" is added to
every expansion. Before this repair, the scope was detached from
module path index shifts that might apply to the bindings (including
references to bulk bindings). Repair the problem by adding suitable
shifts when adding the scope.
Thanks to William Hatch for the bug report.
As suggested by Sam: Using `racket/repl` to start a read-eval-print
loop can mean that less code is loaded if a startup language other
than `racket/base` is selected.
Closes#2064
- add comment saying `check-one-object/equivalent` only compares common
members
- put the similar parts of `check-one-object` and
`check-one-object/equivalent` in a helper function
- in `object/c-equivalent?`, check that names match before comparing the
common contracts (because the names should be fast to check-if-incorrect)
Although the documentation claimed that `read/recursive` produces
a plaeholder, that seems to be a leftover from a much older
reader (before `make-reader-graph`). Fix the new `read/recursive`
to be like the old one, and update the documentation.
Thanks to Alex Knauth for tracking down the unnecessary change
in reader behavior.
Related to #2099
Compared to v6.12, `map` & co. already provide better checking in
reporting an error when a keyword-requiring function is provided
with empty lists, but repair the error message to talk about
required keywords instead of just by-position arity.
Thanks to Philip McGrath for reporting the problem.
Related to #2099
If `local-expand` with a 'module-begin context introduces a macro
definition, but the definition is dropped while a non-macro definition
is later introduced, then make sure references go to the non-macro
definition.
This change also addresses a related scenario, where a
`local-expand`-discovered macro definition is not dropped, but it is
given an extra scope --- which amounts to the same thing from the
expander's perspective.
When a module body is expanded with `local-expand`, then submodules
can remain declared even if the submodule is discarded in the final
expansion. Since that's the way it has always been, leave it that way.
But also guard against a way of generating an import cycle via those
leftover declarations.
the fact that blame object equality now works right and
adding context to a blame object doesn't produce an equal?
blame object
also it appears that blame-add-unknown-context is not actually
being called so lets just get rid of that functionality
(but preserve reasonable backwards compatibility in
case someone is actually calling that function or
supplying #f to blame-add-context)
And the interning of blame objects was not intended to be
in 0b3f4b627e, so get rid of it here
closesracket/typed-racket#722
This ensures macro-introduction scopes don’t unintentionally end up on
lifted pieces of syntax, which causes problems for Check Syntax, since
it affects the syntax-original?-ness of the require spec.
Allow `syntax-local-make-definition-context` in places where the
created scope is not accumulated for stripping from `quote-syntax`.
Refine the docs to clarify those situtations.
A test for the repair exposed a problem with use-site scopes
and `quote-syntax`, so fix that, too.
Closes#2062
When `local-expand` is used for a 'module-begin context, use a fresh
binding -> definition-unreadable-symbol table for the nested
expansion. That way, the table used for the main expansion is
unchanged, and re-expanding or evaluating the expanded module will
arrive at the same unreadable symbols as the initial expansion.
The report and example are from Alexis.
The old implementation turns a single optional argument into two
arguments: the optional value and a boolean to indicate whether the
optional value is supplied.
The new expansion uses `unsafe-undefined` in place of not-supplied
arguments, in the general case. If the default-value expression is
simple enough, however, it is copied to call sites that would
otherwise supply `unsafe-undefined`. In the common case where the
default value is `#f`, for example, no run-time test is needed in the
core implementation function to check whether the default is supplied,
because a `#f` will be filled in for callers.
The performance improvement is tiny to non-existent for realistic
programs, but the simpler and reduced generated code may help in the
long run.
In openssl-1.1 (specifically libcrypto) the functions sk_num, sk_value and sk_pop_free are prefixed by 'OPENSSL_'.
Now both symbol names looked for to support both version 1.0 and 1.1.
The repair in 385f9588f8 propagates the
must-be-bound callback too far. It shouldn't be propagated anymore after
a non-rename transformer is applied.
Closes#2048
A module path index used to expand a module must be interned, and the
intern table is an `equal?`-based weak hash table, which means there's
an internal lock on the table that can be damaged if the current
thread is terminated while using the table.
I don't see an easy way to fall back to `eq?`-based tables, so I'm
resorting to an atomic region (which I had managed to avoid until
now).
* More specific error for no-clause match-lambda** (close#1615)
* Remove unused orig-stx parameter from racket/match internals
* Use of match-XYZ/derived for better errors (fix#1431)
* Tests for the exceptions produced by racket/match
The converstion from calling `call-with-immediate-continuation-mark`
to an internal `with-immediate-continuation-mark` form did not handler
a mutable argument variable.
mzlib/unit200 relies upon this behavior, even though it appears to have
been mostly accidental, so this maintains it for the sake of
backwards-compatibility.
Although splicing was set up for applying a composable
comtinuation to most kinds of continuations, it was not
set up right for applying a composable continaution in tail
position for a just-applied composable continuation.
Thanks to Spencer Florence for the report and example.
This commit adds a section to the reference to document how the expander
tracks information about local bindings, and it extends some
syntax-local functions to allow them to accept multiple definition
contexts instead of just one. In addition, it improves the documentation
on how first-class definition contexts interact with local-expand,
syntax-local-value, and syntax-local-bind-syntaxes, and it also
clarifies what it means to create a child definition context.
If the `#:protocols` clauses of a `define-objc-class` form includes
errors, but it simplifies the declaration of protocols that are
introduced in different versions of a framework, and it's effectively
more compatible with the implementation before dc0898f5ef.
The attempt in 82517622c7 was wrong. Using `JIT_R0` for
the result in the internal ABI is fine, and the problem
was using a register for two purposes in the called
stub.
Redundantly setting the signal handler hasn't matter, but it's
confusing and now matters for implementing W^X via a different signal
handler.
Closes#2038
For consistently with the old expander, treat `#%app` and `#%datum`
like unbound if they're bound to a rename transformer whose identifier
does not untimately refer to macro or primitive syntactice form.
Closes#2042
* match: check duplicate identifiers across list-no-order patters
* match: document that list-no-order doesn't support duplicate ids between sub-pats
* match: put duplicate id docs in a margin note between the two variants
Make 9d0ab74e9e more responsible by limiting permission changes to
pages that are intended to be both writable and executable for code
generation. That way, the signal handler doesn't just reopen holes in
loaded foreign libraries that W^X would close.
A new resource section was aligned based on the old section's
virtual address, instead of the PE's specified section
alignment. That could make alignment round up too far, leaving
a disallowed gap in the sections' virtual addresses.
Conform to W^X by using a signal handler that switches between W and X
mode on any fault. That's not the spirit of W^X, certainly, but it
should make Racket work without special configuration.
Beware that this change can turn some crashes into infinite loops.
It may be possible to detect those loops, but I didn't find a
good and portable way, so far.
Creating an executable with embedded DLLs means that the executable
can be truly stand-alone, instead of needing to be kept with its
DLLs in a relative subdirectory.
DLL embedding works by bypassing the OS's LoadLibrary and
GetProcAddress functions, and instead maps the DLL into memory
and performs relocations explicitly. Joachim Bauch's MemoryModule
(Mozilla license) implements those steps.
Perpetuate a failure to make Windows paths behave reasonably with
path-manipulation functions.
In one case, the new implementation seemed better than the old one, so
I've changed the old implementation (by deleting code) and test cases.
The old code would split "x /y" to "\\?\REL\x " and "y", and the new
one splits to "x /" and "y"; the trailing separator is now enough to
preserve the space character, and it also preserves the directoryness
of the path. Of course, "x /" splits to 'relative and "\\?\REL\x " as
it strips away the trailing "/".
A remaining problem in both implementations: some Windows API
functions implicitly erase a trailing "." in a directory name, making
"x./y" equivalent to "x/y". The Racket path-manipulation functions
don't do that, so splitting and recombining "x./y" does not access the
same path as the original. This apparently hasn't been a problem in
practice, and there are so many terrible hacks already, so I left it
alone.
The new implementation perpetuates also the implementation mistake of
representing paths internally as byte strings. If, in some terrible
universe, I'm forced to do this again, the right choice is probably to
keep the path in a parsed form with enough information to reconstruct
the original, but with the information sorted nicely to make various
normalizations and combinations easy.
Since only the expander is compiled in unsafe mode right now, the
checks are skipped only when the implementations of `reverse`, `map`,
etc., are part of the flattened expander.
Add a cache on binding lookup that is like the old expander --- a
small cache that is consulted before the more general cache that is
already in place.
The new cache layer primarily helps when a single identifier is
compared to a sequence of other identifiers.
The expander as a linklet will be instantiated once, so there's no
need to capture references in closures among functions within the
expander. Add a "static" linklet compilation mode to inline the
variable addresses that would otherwise be referenced via a closure.
Although the change is intended to speed up the expander by avoiding
some indrections, it also reduces the bytecode size of the expander.
Bitmaps that track which linklet variables are used in closures turn
out to have been about 25% of the expander's bytecode size, since the
linklet has so many definitions.
Some parts of the optimizer were inconsistent in whether a tracked
structure type needed to have a constructor that always succeeds
(i.e., no associated guard). Increase precision to track both kinds of
structure types, and avoid some unnecessary space-safety clearing in
the vicinity of nonfailing constructors.
Syntax `[[ "foo" != /* ]]` is not defined by POSIX Shell Command
Language. It's supported only by ksh, Bash and ZSH. Other POSIX
shells, such as ash or dash, does not support it.
This patch replaces this problematic syntax with simple case statement
that is supported by all POSIX-sh compatible shells, including (but not
limited to) ash, bash, dash, ZSH.
This change adjusts the way that trust is threaded through bytecode
and the code inspector. In Racket v6.x, reading bytecode would fail if
the code inspector is non-original and if the bytecode contains a
reference to an unsafe operation. Now, reading bytecode doesn't fail
for that reason, but all bytecode is marked as non-runnable (even
without references to unsafe operations) when loaded under a
non-original code inspector. A `read` operation by itself remains as
safe as ever.
This commit also disables the bytecode validator. For now, the
validate can be re-enabled with `PLT_VALIDATE_LOAD`.
Make `s-exp->fasl` generate an encoding that can be parsed by any
future version of `fasl->s-exp`. The new format does not rely on the
runtime system's bytecode writer and reader.
Some built-in bytes-converter combinations that were not
supposed to require custodian registration were neveretheless
registered, which created a small leak for some programs.
Since the reader's implementation includes quoted references
to infinity and NaN extflonums, make sure it reads and writes
and compiles correctly before "longdouble.dll" is installed.
The `identifier-binding` function doesn't promise to give back any
particular `require` as the nominal binding site, but some tests rely
on the most recent `require` as the binding site. Also, the arrows
in DrRacket look nicest that way.
A change in keyword-argument expansion was intended to avoid
copy-propagation wrok in later passes. That saving does not appear to
be worthwhile, so revert it.
Reverting the change exposed weakness in the expander flattener and an
unsoundness in its simplification pass. That unsoundness has to do
with the assumption that variables are defined before use. The unsound
assumption is likely to be fine for code that is flattened --- all the
more considering that flattened code is routinely run in unsafe mode
--- but it's a departure from the intended safety of the simplifier.
Improving the analysis to so that it's sound and good enough will
require sometimes inferring when a structure-type property guard (for
`prop:evt`, at least) will succeed.
Make the module path index for bindings in a `module->namespace` be a
"self" MPI (with #f for path and base), instead of the MPI associated
with bindings as view from the outside of the module instance. That
makes interactive evalaution in the namespace more closely approximate
expansion within the original module.
Example use: ASL detects a "self" MPI to determine when it should
allow assignment to module-defined variables in the REPL.
Use stdout log reporting instead of stderr log reporting for status
reporting during the build, so that the status report is not
misinterpreted as an error.
This change means that if a macro transformer calls expand (not
local-expand, but top-level expand), the macro debugger won't
receive the inner expand's events. Previously, the macro debugger
tried to parse and then discard the inner expand, but that was
brittle and complicated the grammar.
The old reader used an internal option to short-circuit special-value
reading when a special value acts as a terminating "character". Expose
that shortcut by allowing 'special as an argument to
`peek-{char,byte}-or-special`, and update the reader to use it.
This commit merges changes that were developed in the "racket7" repo.
See that repo (which is no longer modified) for a more fine-grained
change history.
The commit includes experimental support for running Racket on Chez
Scheme, but that "CS" variant is not built by default.
Thanks to Philip McGrath for spotting the problem
Also, along the way, discover the setup for chaperoneness for opt
contracts was bogus, so fix that up too
Handling of runstack overflow wasn't right for platforms
with a distinction between short and long jumps --- which
includes x86_64, but only after enough code has been
allocated.
The previous definition of define-simple-macro would fail (with
a very poor error message) if the user made a simple macro like:
(define-simple-macro (~or a b)
(let ([tmp a])
(if tmp tmp b)))
While the define-simple-macro form does allow syntax-parse patterns,
this should not be enabled for the initial head pattern, as described
in the define-simple-macro documentation.
Reduce the generation of foreign-function wrappers by `ffi/objc`
by caching type vectors when all of the types refer to module-level
constant bindigs.
Includes an optimizer-validation repair, where improved generation
creates a reference to a variable that would normally be replaced
by constant propagation.
The `#:blocking?` argument has no effect for now, but it will enable
better cooperation with OS thread in Racket-on-Chez. Function that
block indefinitely in a place, for example, will need to be called
with `#:blocking? #t` to prevent stalling GCs in other places.
Add a `_bytes` variant type that will work more consistently with
Racket-on-Chez, where the representation of a byte string does not
include an implicit nul terminator.
Faster for the case that an impersonator has a small number of
impersonator properties. To enable hand-coded search, a small number
of properties are now kept in a vector instead of a hash tree.
Commit 89512eda had accientally disabled the JIT's fast
path for structure-type property predicates and selectors.
Re-enable it, but but repair a problem with the way that
impersonator-property operations are handled by the same
code.
Before 544b7a3d53, an incorrect barrier check prevented applying
a continuation captured in other threads from being applied in a
thread created with `call-in-nested-thread`. There's no good
reason to prohibit such an application, and now that the bogus
check is gone, an explicit prompt is needed in the implementation
to make it behave correctly.
Since the previous version of this code was first written, Racket's
requirements seem to have relaxed---but not completely!
If *all* related module paths are made relative, then the deps
test case breaks (but only the test is pre-compiled ?!!).
If requires from disappearing module(s) to residual module(s)
use absolute module paths, it seems to work.
The dance that syntax/parse performs to lazily load its implementation
rewrites uses of syntax/parse macros in such a way that their original
names were discarded, which shows up in error messages. By simply
invoking the underlying transformer directly in the proxy macro instead
of expanding to a rewritten use, users’ names can be preserved.
fixes#1909
Don't report ltps success for file descriptors that are not supported
by epoll.
Racket ports probably were not affected, since Racket doesn't try ltps
registration for regular files or devices like /dev/null that report
always being ready.
Provide unsafe functions for working with file descriptors and
sockets. Although more functions are potentially useful, these
reflect the one scurrently exported by the C API.
When a composable continuation has continuation marks that should
be merged with marks in the immediate continuation frame when
the continuation is applied, then mergeing did not always work.
It only worked in the case that the merge cadidates are the only
marks, because the merging check used the wrong end of the captured
sequence of marks.
Note: this version doesn't work with ~commit or ~!, because
it stores both choice points and undo actions in the failure
continuation. Commit and cut should discard choice points but
preserve undo actions.
Repairs another regression due to the rktio conversion, where a
connection that takes a while can be mismanaged, especially if
multiple connection attempts run concurrently.
Using `(local-expand <expr> 'module-begin <stops> def-ctx)` didn't
work right, because definitions added to `def-ctx` were not visible.
(While adding definitions before `module-begin` expansion is an unusual
thing to do, there's no reason that it has to fail.)
Repairs a problem with ce9894c8bf, where a large "inlined" vector
allocation is not actually inlined, but other parts of the JIT
assume that it will behave as inlined --- which implies that the
runstack will be left unchanged after the call.
Closes#1868
In Windows 10 Fall Creators Update, a OneDrive mount looks to Racket like
a reparse point that refers to itself. Make Racket stop trying to resolve
the path further in that case.
Relevant to #1671
Commit 00d438cfbe made an attempt at this,
but this commit does it in a much more careful way, based on manually
emulating how the macroexpander expands module* forms in order to allow
splicing-syntax-parameterize to apply even within #%module-begin forms
introduced by the expander.
Mostly, fix "compiled" versus "expanded" in an error message,
but this variant of the message copied from the new expander
implementation seems better overall.
Repairs a problem reported by Laurent Orseau, where running the
following program with errotrace enabled reports an error:
#lang racket/base
(require racket/class)
(new object%)
(define-namespace-anchor nsa)
(define ns (namespace-anchor->namespace nsa))
(eval '(new object%) ns)
I haven't been able to make a standalone test, yet, but I'll keep
trying.
Normally, it's impossible to generate lots of `eq?`-hashing
collisions, but when the compiler inlines a function, it can duplicate
variables in a way that gives each copy the same `eq?` hash code. The
immutable-hash tree implementation failed when more than 32 collisions
occurred (which triggers a subtree in the collision node).
It's similarly very difficult to generate > 32 values that collide on
`eqv?` hashes but are not `eqv?` (although it must be possible using
exact rationals or complex numbers).
On my machine, this reduces the running time of the sqlite3
tests by about 1/4 (~3.2s to ~2.4s).
Other things I tried that didn't make as big a difference:
- coalesce A regions
- fast path for call-with-lock
Note: quasisyntax has a bug: #`(... (1 2 #,@(list 3) 4)).
Within an escape, no way to express splicing desugaring.
So add a private variant of ?@ that is interpreted even escaped.
Make datum->syntax explicit in guide rather than combined with
constructors like t-cons/x and t-dots (conditional).
This will make datum support easier, later.
For now, it makes it easier to do relocate correctly.
Also, make t-metafun, h-splice inlinable.
Although `raco pkg` doesn't use a package's ring number, it's useful
to preserve for other tools (like the pkg-build service). Adjust `raco
pkg catalog-copy` and `raco pkg catalog-info` to recognize and store a
ring number.
A mistake in the rktio conversion causes a crash if certain functions,
such as `directory-exists?`, are used before certain other functions,
such as `resolve-path`.
Thanks to Alex Harsanyi for the report.
The rktio conversion lost the deregistration of file descriptors in an
internal fd-to-semaphore table building on kqueue/epoll, causing the
wrong semaphore to be checked for a later recycling of the file
descriptor. This bug mainly affects Linux and ports created by
`subprocess`, since kqueue is not used for pipes on Mac OS and BSD
variants. The bug does not affect network sockets (which are the
primary intended clients of epoll/kqueue support), since the relevant
semaphore is deregistered when a socket is closed.
Thanks to James Bornholt for discovering the problem and providing the
repair.
Closes#1769
Fix a problem with compile-time bindings added to a
namespace created by `module->namespace` for a module
that does not have a source file.
Possibly, there's a different fault that should be fixed that caused a
binding to use the module's instantiation-time module path index
instead of its compile-time module path index (which is what happens
when a file is involved). This repair fixes the problem in a general
way, though, and leaves further improvement to the reimplementation of
the expander in Racket (which already does not suffer from the bug).
Thanks to Alexis for providing the example.
In a non-tail position, a JIT-generated application of `apply`
retained the argument list until the called function returned.
Fix it to drop the reference to the list before the function
is called.
And unquoted-printing string contains a string to `display` in all
print modes. Although it could be implemented with a structure type
that has a printing function, `raise-arguments-error` further treats
unquoted-printing strings specially by not using the error value
conversion handler, so it reliably produces literal text in the error
message; that way, `raise-arguments-error` can be used to construct
more error messages.
The hack to implement `_union` without help from libffi failed when
the total size of the variants is too large. Try a different approach,
which involves a bet that the total size plus whether the content is
all floating-point numbers will be enough information for most cases.
Relevant to #1351
The switch to rktio swapped the sense of "signal" and "sigset"
incompletely, and "signal" on Solaris is the one that resets
the signal handler to SIG_DFL.
Make `raco setup` propagate the original value of
`use-compiled-file-paths` in case it's reset to avoid loading bytecode
files. Then, `--clean` mode can remove bytecode relevant to that
setting, instead of always clearing "compiled" subdirectories.
There's no external way to initialize `use-compiled-file-paths` right
now, other than forcing it to `null` with the `-c` flag at the
`racket` level, but the current "racket7" implementation uses
different `use-compiled-file-paths` settings for different build
modes, and it seems to make sense in general.
Also, make `--clean` sensitive to `-D` and `-d`, so that it's easy to
clean just bytecode.
The copy was made in c1c427a281 to help get "racket7" started, but
it's no longer needed. Using the copy had an unintended consequence of
duplicating the collection-search cache for phases other than phase 0,
so this change repairs that consequence.
The `racket/private/check` module is dead code in this repo, but
having it simplifies the "racket7" bootstrap when "racket7" goes so
wrong that it can't rebuild itself.
To strip a signature, the old implementation effectively guessed
at the padding that was added to the original linkedit segment
to acommodate 16-byte alignment of the signature. The repaired
stripping works out the actual end of the linkedit segment based
on the various load commands that refer to it.
Add guard, if first argument to `find-relative-path` is not a path,
then don't call `path-convention-type` on it.
Thanks to Jack Firth for reporting:
https://github.com/racket/rackunit/pull/46
The cause is any extra space after a terminating semi-colon in a SQL string passed to sqlite3_prepare_v2 procedure in the prepare1* method of `db` Sqlite3 connection% implementation will result in non-false tail and zero prep-status values being returned. The checking code sees the non-false tail value and raises the "multiple statements given" exception.
The easy fix is to trim any trailing whitespace by calling string-trim on the buffer string before its converted to a byte string in the copy-buffer procedure used by sqlite3_prepare_v2 to initialize the sql-buffer.
See Issue #1702
The same problem is encountered when executing a DDL statement such as "Create Table" more than once after the table had already been created by the first execution.
The cause is when sqlite3_prepare_v2 encounters a bad statement, it aborts the preparation, sets prep-status flag to non-zero and also sets tail? to be non-false.
The previous code ignored the prep-status flag and only checked tail? and raises "multiple statements given" exception when tail? is non-false.
The new code takes prep-status into account and extracts the actual Sqlite3 error message by evaluating get-error-message and raising it as an exception.
See Issue #1702
Also, add a script to extract the "rktio.h" header information to an
S-expression form suitable for FFI use, and refine the "rktio.h"
information to improve that output.
Avoid printing `(unquote @d)` or `(unsyntax @d)` as `,@d` or `#,@d`,
which would mean `(unquote-splicing @d)` or `(unsyntax-splicing @d)`
to the reader, by adding an extra space before the `@`.
When a package to be installed has a module that conflicts with an
existing installed module, but the existing module isn't in a package,
then say so explicitly in the error message. Also report the current
collection and links paths.
Error reporting could do even more work to figure out which path is
relevant, but reporting a list of paths is relatively easy. Listing
paths also seems likely to give a user enough hints for this rare
case, especially if the user just needs to be reminded that collection
links exist.
Additionally this adds macros that distinguish between the chaperone redirects of prop-only vector
chaperones, function chaperones, and structure chaperones since each of these may store a vector
in the redirects field.
Auto fields were incorrectly recorded as immutable in a structure type
that is first generated from the prefab struct key instead of
`make-struct-type`.
Thanks to Deren Dohoda for the report.
When `--update-deps` (which is implied by `--all`) was used on
user-scope packages that depend on installation-scope packages, the
installation-scope dependencies were treated as missing, which forced
an update of the user-scope packages. Check both scopes for
dependencies.
Closes#1730
The `file/cache` relies on matching the text of a message, and the
text had changed to have less information on Unix, including whether
the problem was with the source or destination file. Add a notion of
error step to rktio and use it to improve the `copy-file` message.
Report a proper error when path decoding fails, instead of
synthesizing a path hat shouldn't exist. The rktio conversion
made it much easier to report the error at the Racket level
like other filesystem errors.
In the process of extracting minimal Windows path encoding for rktio,
I noticed a decoding issue with a path that ends with an unpaired
high-surrogate value. Add a suitable tests and fix the old decoder
(although it will probably go away).
Closes#1721, which points out that the `for/list` expansion
introduced in commit 5e94a906cd interacts badly with a body
that captures a continuation plus Racket's current implemenation
of continuations.
When Racket one day gets a better implementation of continuations,
this change could be considered again, but the general question is
whether programs can detect or be affected by the size of the
continuation (when the programs don't directly control the
continuation creation --- otherwise continuation marks obviously
expose the size).
Provide a compatibility layer to bridge the old (not completely
documented) `scheme_fd` interface and rktio. The old interface is
used by the Gtk implementation of `racket/gui`, for example.
Process handling relies on SIGCHLD being blocked, but signals are
blocked per-thread in Linux, so SIGCHLD needs to be blocked before new
threads are created.
For `open-output-file`:
Move 'replace mode handling out of rktio and into the client.
Formerly, on Windows, 'replace mode was just 'truncate with a fallback
for permission problems, so this change makes it delete an existing
file and replace it, which is more consistent with how 'replace has
always worked on Unix.
In 'truncate/replace mode, if a truncating open fails due to a
permission error and the file exists, then try again as a 'replace.
That's how it worked on Windows before, and now it's how Unix works.
The world doesn't need yet another cross-platform I/O library, but
it's getting one. This one has exactly the things that Racket needs,
and pulling it out will make it reusable from other VMs while
improving the Racket code organization.
This first step just gets started.
For a term
(lambda (arg-id ...) (define def-id _rhs) ... (arg-id def-id) ...)
the expander could take quadratic time in the number of `def-id`s
due to walking an environment to remove use-site scopes. (The
variant of the expander rewritten in Racket didn't have this
problem.)
This commit add mzrt_sema_try_wait to the functions that operate
on semaphores. The existing ones are:
* int mzrt_sema_create(mzrt_sema **sema, int init);
* int mzrt_sema_post(mzrt_sema *sema);
* int mzrt_sema_wait(mzrt_sema *sema);
* int mzrt_sema_destroy(mzrt_sema *sema);
This make-c-id allows an author to specify a convention for how
to connect and identifier defined with define-ffi-definer and
the actual symbol in the file.
* Adds docs.
* Adds tests.
* Adds history.
Accessing unsafe functionality through the FFI seemed like a good way
to avoid writing C code, but it made things more complicated instead
of easier, and it interacts badly with a more agressive shift away
from C (such as porting to Chez Scheme). So, add functions to the
primitive `#%unsafe` module, instead.
An authentic structure type is one whose instances cannot be
impersonated or chaperoned. The intended use of `prop:authentic` is to
annotate a library-private data structure where impersonators are
never needed internally for the data structure, and the declaration
lets the compiler produce less code and fewer branches by omitting
impersonator support.
In non-cross mode, `-C` needs to go after `-G` and `-X` when setting
up a "bundle" directory to turn into an installer, because that mode
needs to use foreign libraries (such as SQLite) at build time, and it
can use the instances that are being set up for the installer.
Meanwhile, improve the advice for setting `PLAIN_RACKET` to use `-C`
for a cross-platform build mode, even though things tend to work
anyway without it.
When comparing a part of a hamt that is a collision node versus a
subtree node, a "hash code" was extracted from the collision node ---
but that's really a code for an integer key is that used for the
collision element. The comparison should instead use a code extracted
from the reference to the collision node (which is the code that is
common to all colliding keys).
Detangle the target and host DLL and library directories by
making `get-lib-search-dirs` and `get-dll-dir` report the
host system's directories, and add `get-cross-lib-search-dirs`
and `get-cross-dll-dir`.
A new `-C`/`--cross` flag causes `racket` to save a host config and
collection directory and make them available via `(find-system-path
'host-{config,collects}-dir)`, while plus `(system-type 'cross)`
reports whether `-C` mode is in effect. Besides making the host paths
available, this change allows a same-platform build to run in
corss-platform mode.
The immediate problem to solve was the creation of Windows installers
on Windows, where recent changes to support 'gui-bin-dir configuration
need a clear distinction between the host Racket and the target Racket
being built, even if they're the same platform. (The "GRacket.exe"
executable didn't work, for example.)
The changes in this commit are more than needed for the immediate
problem, but they naturally build on the necessary `-C` flag, and they
support cross-platform package setup where native libraries are needed
during setup.
Avoid the well-known possibility of quadratic handling of ephemeron
chains, where all ephemerons are immediately known, no keys are
immediately known, and each link in the chain has a value that refers
to the next link's key.
To aviod quadratic behior, attach a list of ephemerons to each page of
allocated objecst, where marking any object on the page triggers a
rescan of the ephemerons without waiting to rescan all ephemerons.
Make `log` in `racket/base` optionally accept a second argument.
The second argument is the log `base`. The docs also recommend
`fllogb` when precision is important.
* Error message when base is 1
* Added docs.
* Add tests.
Store relative paths in "info-cache.rktd" (which corresponds,
roughly, to packages) in a platform-independent form, instead
of using the current platform's convention.
Using the current platform's covention works badly when
cross-compiling for Windows on Unix, since relative paths are used as
keys in the "info-cache.rktd" table. For example, updating a
pre-installed package on Windows mangles the mapping if the installer
is created from a cross-compiled installation.
Optimization to convert `(hash-ref <ht> <key> (lambda () <constant>))`
to `(hash-ref <ht> <key> <constant>)` didn't check that the `lambda`
for had zero argument.
Closes#1648
* generics: optional scope arg for private macros
To make them more friendly to macros that expand to generics
* add tests for generic-method-table macro
and/c, and between/c (which implies <=/c and >=/c) so that they turn
themselves into integer-in when appropriate
for example, (contract-stronger? (integer-in 0 4) (and/c natural? (<=/c 4)))
returns #t
- positive-integer?
- negative-integer?
- nonpositive-integer?
- nonnegative-integer?
These are like their exact-* counterparts provided by racket/base,
but they work for inexact numbers, and not just exact ones.
Fix problem with once-use tracking and delayed variable-use marking
that is performed for local function bodies. A delayed variable-use
registration might happen after a once-used variable is replaced by
its use.
This scenario is difficult to provoke, because the optimizer has to
first decide not to move a once-use function, and in a latter pass
decide to move it after all. There's not enough information to
retract the tentative use plus its transitive implications.
The solution is to avoid the generic once-use layer for `lambda` forms
whose uses are delayed (and that likely has a good effect on inlining
anyway). The other half of the solution is to avoid transitive use
marking on a once-used variable whose expression has been moved (and
there are no transitive things to skip, because that expression isn't
a `lambda` form).
It appears that Mac OS wants `RTLD_LOCAL` in the dlopen() call,
otherwise dlsym() searches through all previously open shared objects
– even though dlopen() is given a specific library handle.
The expr/c syntax class, as well as its underlying implementation
function, wrap-expr/c, previously produced misleading error messages.
The main purpose of these tools is to ensure a user-provided expression
conforms to a macro-provided contract. However, contract errors produced
by these forms were consistent with situations where both value and
contract were provided by the same party.
This fixes the discrepancy by changing how these forms assign blame to
emulate contract errors that arise from improper function arguments,
since most expressions provided to macros are semantically similar to
function arguments. All examples within the documentation itself
reflect this use case.
These changes alter the contents of error messages raised by expr/c and
wrap-expr/c, which could theoretically break some test suites, but it’s
extremely unlikely that any non-test code would depend on the precise
wording of contract error messages, and the interface is otherwise
completely backwards-compatible.
fixes#1412
When the second argument to `bytes-set!` is a reference to a
module-level variable that is definitely defined but not a known
constant, then an incorrect reordering was used that would cause
the third argument value to get overwritten before the call.
Closes#1601
The primitive `read` uses a shortcut --- a private "ungetc"
implementation --- that did not count position correctly for
non-ASCII characters.
Closes#1599
The documentation says that it should work on any output port,
although there's special treatment of ports that originate
from `pretty-print` itself.
Closes#1579.
Allow the directory for GUI executables to be specified as different
from console executables. The defaults for those two are different
on Mac OS, and configuring them differently might be useful to
address #1575.
Although there is probably no demand on Windows or Unix for splitting
the console and GUI bin directories, this patch tries to make things
work sensible there. On Windows, there's a corner case where a
launcher that starts GRacket (especially with `-z`) is intended to be
a console executable. The launcher creator can be told that via a
`subsystem` option, but a new `#:console?` argument was needed for
`make-gracket-launcher-path` lets the path selector know.
After some reductions, the new rator advance less the effect
clocks than the original rator. For example in
(equal? x 7) ==> (eq? x 7)
(my-struct? x) ==> #t or #f
The lambdas can be marked as single valued and/or mark preserving.
With this information is possible to remove unnecessary wrapping
like the `values` in
(let ([f (lambda () '(1))])
(display f f)
(values (f)))
or in reductions like
(car (list (f))) ==> (values (f)) ==> (f)
Moreover, this is useful to test that the optimizer has marked
correctly the function f as single valued and mark preserving.
If a module has any sort of complex bindings, such as a definition of
a macor-introduced identifiers, then `module->namespace` and variants
(like `variable-reference->namespace`) need to recreate suitable
bindings. Make sure that the module-path index for recreated bindings
is the run-time one, not the compile-time one.
Closes#1584
To avoid moving expressions that may have a side effect, the optimizer must
recognize that in this position this will cause an error and advance
the virtual clock.
Currently the only primitive that is flagged as SCHEME_PRIM_IS_OMITABLE and
may have multiple return values is `values`.
Thanks to Robby for finding the original version of the test.
When a hash table or other special value appears immediately on the
right-hand side of `define-values`, it needs to be protected by an
explicit quote when writing to bytecode.
Closes#1580
* byte-regexp? values should not be considered 3D syntax.
* hash? values are now allowed in serialized syntax properties with (template … #:properties (…))
* marshalling properties which were prefab structs called map on the result of struct->vector, changed it to struct->list as the struct "name" is always serializable.
Continuing the saga that includes 8190a7730d and d1ba9fbb6e, it turns
out that a 0-binding clause as the last one isn't so special after
all. A little later in the optimizer, now that we're sometimes moving
an error to the body, we can't assume that the body can be discard
if an error was detected.
Set up bindings and shift phases as needed to make
`variable-reference->namespace` work in a run-time position when the
enclosing module is instantiated at a phase other than 0.
Thanks to Rohin Shah for the bug report.
Support an external implementation of `read-syntax` by exposing
functionality that is currently internal to `read-syntax`: a srcloc
argument to a "special"-producing port function and wrapping special
results to reliably distinguish them from characters.
When Racket is run with stdout or stderr redirected to a file,
then it must be treated as a regular file, otherwise flushing
and position counting doesn't work right.
Merge to v6.8
In e59f888, new GCs no longer inherit the `avoid_collection` value set by
PLTDISABLEGC, and so that setting is lost as soon as the master place spawns
(due to `GC_switch_out_master_gc`).
When multiple-binding `let-values` form is split into a single-binding
form on the grounds that the right-hand side will definitely error,
the optimizer's effect clocks were advance incorrectly.
Closes#1552
Instead of duplicating a context line, show "[repeats <n> times]".
This improvement particularly helps avoid showing less context
now that `for/list` creates a non-tail recursion to build up
the result list.
Specifically, when it sees these contracts:
(and/c real? negative?)
(and/c real? positive?)
(and/c real? (not/c positive?))
(and/c real? (not/c negative?))
it generates the corresponding use of >=/c, <=/c, </c, or >/c, but
those contracts have also been adjusted to report their names as
(and/c real? ...).
This mostly is an improvement for contract-stronger, but also make it
so that (between/c -inf.0 +inf.0) just uses the real? predicate
directly, instead of a more complex function
Although "macOS" is the correct name for Apple's current desktop OS,
we've decided to go with "Mac OS" to cover all of Apple's Unix-like
desktop OS versions. The label "Mac OS" is more readable, clear in
context (i.e., unlikely to be confused with the Mac OSes that
proceeded Mac OS X), and as likely to match Apple's future OS names
as anything.
For example, an `unsafe-unbox` call should not be moved past the
call to an unknown function that might change a box's content.
Thanks to Sergey Pinaev for the report.
Using `--disable-jit` causes futures to be disabled, and places are
always disabled for CGC; in that case thread-local variables are
not needed. Meanwhile, the 3m build still has places, so a "gmp.c"
compiled without thread-local support is broken.
The objective of lookup_constant_proc and the first part of
optimize_for_inline was to find out if the value of an expression was a
procedure and get it to analyze its properties or try to inline it. Both
were called together in a few places, because each one had some special
cases that were missing in the other.
So, move the lookup and special cases from optimize_for_inline to
lookup_constant_proc, and keep only the code relevant to inlinig in
optimize_for_inline.
If an OS-level thread other than a Racket thread logs a message, then
the message needs to be queued instead of handled immediately.
If multiple places are running, then the right handler thread is not
clear, so just queue to the main place's thread.
Closesracket/gui#66Closesracket/drracket#77
Implement POSIX.1-2001/pax and GNU extensions for long paths and links
in `untar` and `tar`. Add a `#:format` argument to `tar` to select
among POSIX.1-2001/pax, GNU, or error encoding for long paths.
it doesn't apply it the second time (since we know that the
only difference for indy blame is in the negative position
and we know that flat contracts never assign negative blame)
This commit combined with the two previous (2b9d855 and 003e8c7) do
not seem to have a significant effect on the performance of ->i
contract checking. In particular, I see a 50% slowdown between the
version before and the version after these commits on the third `time`
expression below, but no significant difference on the first two.
(without the improvement to flat-contract?, these commits are
a significant slowdown to `g`)
#lang racket
(require profile)
(define f
(contract (->i ([y () integer?]
[x (y) integer?])
(values [a () integer?]
[b (a) integer?]))
values
'pos 'neg))
(define g
(contract (->i ([y () (<=/c 10)]
[x (y) (>=/c y)])
(values [a () (<=/c 10)]
[b (a) (>=/c a)]))
values
'pos 'neg))
(define (slow-predicate n)
(cond
[(zero? n) #t]
[else (slow-predicate (- n 1))]))
(define h
(contract (->i ([y () slow-predicate]
[x (y) slow-predicate])
(values [a () slow-predicate]
[b (a) slow-predicate]))
values
'pos 'neg))
(time
(for ([x (in-range 100000)])
(f 1 2) (f 1 2) (f 1 2)
(f 1 2) (f 1 2) (f 1 2)
(f 1 2) (f 1 2) (f 1 2)))
(time
(for ([x (in-range 100000)])
(g 1 2) (g 1 2) (g 1 2)
(g 1 2) (g 1 2) (g 1 2)
(g 1 2) (g 1 2) (g 1 2)))
(time
(for ([x (in-range 10000)])
(h 50000 50000)))
When the main interpreter loop is called for an application where the
argument array coincides with the current runstack pointer, then the
protocol is that the callee gets to modify that space --- and it
should modify that space as arguments become unused. The interpreter
was always copying arguments to a fresh space, though.
Both function have a similar purpose and implementation, so merge them to consider
all the special cases for both uses.
In particular, detect that:
(if x (error 'e) (void)) is single-valued
(with-continuation-mark <chaperone-key> <val> <omittable>) is not tail sensitive.
Also, as ensure_single_value was checking also that the expression was has not a
continuation mark in tail position, it added in some cases an unnecessary
wrapper. Now ensure_single_value checks only that the expression produces
a single vale and a new function ensure_single_value_noncm checks both
properties like the old function.
Adjust list and stream handling as sequences so that during the body
(for ([i (in-list l)])
....)
then `i` and its cons cell in `l` are not implicitly retained while
the body is evaluated. A `for .... in-stream` similarly avoids
retaining the stream whose head is being used in the loop body.
The `map`, `for-each`, `andmap`, and `ormap` functions are similarly
updated.
The `make-do-sequence` protocol allows an optional extra result so
that new sequence types could have the same properties. It's not clear
that using `make-do-sequence` is any more useful than creating the new
sequence as a stream, but it was easier to expose the new
functionality than to hide it.
Making this work required a repair to the optimizer, which would
incorrectly move an `if` expression in a way that could affect
space complexity, as well as a few repairs to the run-time system
(especially in the vicinity of the built-in `map`, which we should
just get rid of eventually, anyway).
Compile a `for[*]/list` form to behave more like `map` by `cons`ing
onto a recursive call, instead of accumulating a list to reverse.
This style of compilation requires a different strategy than before.
A form like
(for*/fold ([v 0]) ([i (in-range M)]
[j (in-range N)])
j)
compiles as nested loops, like
(let i-loop ([v 0] [i 0])
(if (unsafe-fx< i M)
(i-loop (let j-loop ([v v] [j 0])
(if (unsafe-fx< j N)
(j-loop (SEL v j) (unsafe-fx+ j 1))
v))
(unsafe-fx+ i 1))
v))
instead of mutually recursive loops, like
(let i-loop ([v 0] [i 0])
(if (unsafe-fx< i M)
(let j-loop ([v v] [j 0])
(if (unsafe-fx< j N)
(j-loop (SEL v j) (unsafe-fx+ j 1))
(i-loop v (unsafe-fx+ i 1))))
v))
The former runs slightly faster. It's difficult to say why, for
certain, but the reason may be that the JIT can generate more direct
jumps for self-recursion than mutual recursion. (In the case of mutual
recursion, the JIT has to generate one function or the other to get a
known address to jump to.)
Nested loops con't work for `for/list`, though, since each `cons`
needs to be wrapped around the whole continuation of the computation.
So, the `for` compiler adapts, depending on the initial form. (With a
base, CPS-like approach to support `for/list`, it's easy to use the
nested mode when it works by just not fully CPSing.)
Forms that use `#:break` or `#:final` use the mutual-recursion
approach, because `#:break` and #:final` are easier and faster that
way. Internallt, that simplies the imoplementation. Externally, a
`for` loop with `#:break` or `#:final` can be slightly faster than
before.
When parsing cannot fail, avoid allocating expectstacks and
failures (thanks samth for the idea). Allocation still happens
for progress and failuresets (conses of #t, now), though.
Compile with `PLTSTDERR="debug@syntax-parse"` to log cannot-fail
syntax-parse expressions and syntax class definitions.
The optimizer can detect that some expressions will escape through
an error, and it can discard surrounding code in that case. It should
not change the tailness of a `with-continuation-mark` form by
liftng it out of a nested position, however. Doing so can eliminate
stack frames that should be visible via errotrace, for example.
This change fixes the optimizer to wrap an extra `(begin ... (void))`
around an expression if it's lifted out of a nested context and
might have a `with-continuation-mark` form in tail position.
When caching the result of a stack traversal, adjust
the actual stack only after the traversal is complete
to avoid interfering with libunwind's decoding of the
stack.
In
(with-syntax ([x ....])
#'(x y))
and property on the source syntax object `(x y)` was lost in
constructing a new syntax object to substitute for `x`, while
properties on preserved literal syntax objects, such as `y`
were intact. Change `syntax` to preserve properties for
reconstructed parts of the template.
This change exposes a problem with 'transparent taint modes,
where the internal "is original?" property was preserved while
losing scopes that wuld cancel originalness. So, that's fixed
here, too.
