Generalize splitting of `(let-values ([(x ...) (values e ...)]) ....)'
to `(let ([x e] ...) ....)' for any `e', since it's always equivalent.
Right?
(The old requirements on the `e's seem to be needed only for
`letrec-values' splitting and maybe mutable variables.)
Treat unsafe functional operations (which never raise an
exception) as omitable, which means that simple `let-values'
combinations can be split into `let' bindings, etc.
corner of the definitions window, based on the information that check
syntax computes
This commit contains two separate changes to make this work:
- adding a new renderer, based on the text renderer, that
pulls out the contents of the blue boxes and saves them
in the doc/ directories (specifically in the files named
contract-blueboxes.rktd)
- extend check syntax to use and display the information
build by the new renderer
Internally, there's a `prop:method-arity-error' property that is
used for keyword-accepting methods. The same thing could be
accomplished with `procedure->method', but the new property avoids
a wrapper. It might be nice to expose the property from `racket/base',
but that creates trouble for generating arity errors for keyword-
requiring procedures (i.e., when such a procedure is wrapped), so
keep it provate for now.
Closes PR 12982
A progress evt from a close input port must be initially ready,
and the primitive `peek-bytes-avail!' checks a progress evt
before checking whether the port is closed.
These changes resolve a race in `read-bytes-evt' and related evt
constructors.
Fix tchecking for a rest argument to a function that
is lifted by closure conversion so that one of its
arguments is a mutable local variable's location.
Also reject bytecode that would pass too many arguments
to a lifted function, since that would trigger an arity
error that might try to use a location as a value.
Merge to v5.3
When a `port-commit-peeked' succeeds, position information should
(appear to) be updated. This patch synchronizes commits and
position information for primitive ports, but synchronizing
them for user ports remains a problem.
When `guard-evt' or `nack-guard-evt' is used, `sync' didn't
account for the possibility that a channel or semaphore
action might complete during the call to the guard, in which
case it might fail to select the event that has already
completed.
Merge to v5.3
Convert
(hash-ref <hash> <key-expr> (lambda () <literal>))
to
(hash-ref <hash> <key-expr> <literal>)
which is useful for making the `case' expansion fit
Typed Racket.
The "simpliciation" of a syntax object's lexical context was dropping
module contexts that have no bindings, but those contexts now
contribute to the identifty of some bindings. Fix simplification
to replace the full rename rename with a simplified one, instead
of just dropping it.
Merge to v5.3
When a module is loaded from bytecode and then the value of
`use-compiled-file-paths' changes, an attempt to load a submodule
would fail, because source isn't used if the main module is
already declared, and the bytecode code is not used according to
`use-compiled-file-paths'. Make the bytecode path stick when it
is used once, so that submodule loads succeed, and make it work
even with `namespace-module-attach'.
The module-attach part of this protocol requires a change to the
API of a module name resolver: the notification mode gets two
arguments, instead of one, where the second argument is an
environment.
The lambda-lifting transformation needs to iterate to a fixpoint
where each lambda's added arguments and order are known. The
check for whether something changed was formerly just the number
of added arguments, but that's not good enough, because a binding
might get lifted away while another one acquires an extra argument.
The right test is to check the count and original bindings for the
added arguments.
Closes PR 12910
Since the optimizer can lambda-lift a function and adds box arguments
(i.e., locations of mutable variables instead of values), then
the validator must check that calls and declarations are consistent.
But declaration-side information wasn't registered properly.
This is an old bug, but it was exposed by recent improvements
in unboxing.
Also, fix JIT implementation of explicitly decremented fuel on
a 64-bit platform, plus some other code clean-up.
Each future thread has its own x87 flags, apparently reset to the
default (at least on Mac OS X) rather than inherited from the
creating thread, so reset the x87 configuration in each new future
thread.
To support module caching, module path indexes must be cloned
for each use of the cached module, so that path resolutions
don't collide. Cloning was previously implemented at the point
of shifting the modidx based on the module instantiation name,
but now its cloned at declaration time. This result in better
sharing of module-path resolutions, which in turn speeds up
compile-time instantiation of modules, which in turn speeds up
interactions & examples in documentation (as much as 10% for the
Guide, for example).
Furthermore, the reverse cache within a modidx may have been
used improperly during cloning, and that's not a problem in
the new implementation.
The JIT wants to detect calls to functions bound to module-level
variables that are always instantiated to a procedure of a particular
arity, in which case it can avoid some checks. The problem is that
bytecode and JITted code can be shared via the module cache across
namespaces that use different modules to implement a particular module
path or that enable the JIT differently. In particular, starting with
one that has a procedure binding and then using one (in a different
namespace) that has a non-procedure binding could lead to a crash.
Defend against this possibility by never treating imported variables
as constant in that sense. The JIT detects imported variables through
a new import map in the prefix.
This change may slow code. My guess is that it will have no
performance impact in practice, due to cross-module inlining,
although I can construct a microbenchmark that slows by 20%.
The optimizer can no longer reduce
(with-continuation-mark _id _v-expr _expr)
to just
_expr
when _v-expr and _expr are simple enough, because _id
might be bound to a continuation mark key with an
impersonator that checks the result of _v-expr.
The loss of an optimization can have a significant affect on
errortrace of microbenchmarks, such as
(for ([i (in-range 10000000)])
i)
A arity-reduced procedure-valued `prop:procedure' was not handled
correctly, for example.
A good candidate for random testing? I had the right kind of test in
place, but only for an arity of 1. It turns out that testing any other
arity would have exposed the problem, so now there are tests with
arity 0. If I had randomly generated procedures instead of manually
constructing `f0' through `f1:+' in "procs.rktl", then maybe I would
have more naturally generalized the arity testing, too. Then again,
I did already have relevant inputs, and it was the testing of inputs
that was too specific.
Closes PR 12870
For all currently supported platforms, the result was already
portable, despite the documentation's hedging.
Also fixed up the documentation in other ways, such as the fact
that `seconds->date' returns a `date*'.
Add extra intitial-message lines, use "..." on a field name
to indicate that it could reasonably be hidden by default,
and refine some existing messages.
Includes the addition of 'overflow and 'start-overflow-work
events, whcih are effectively specializations of 'sync and
'start-work to expose overflow handling.
Also, fix a bug related to a potential GC during mark-stack
restore from a lightweight continuation.
By itself, this change won't help anything, because tail-call
allocation is triggered for something that can't be called directly.
This change sets up part of an improvement for future-local recovery
from stack overflow, though.
(I had trouble constructing a test that would trigger the new code.
Fortunately, the existing tests trigger it.)
When a "synchronized" operation is handled for a future thread
during a `touch' on the future, then still report the handling
as "synchronized" insteda of "blocked".
Also, fix FFI procedures to preserve names: change `ptr-ref' with
`_fpointer' on an `ffi-obj' value to return the `ffi-obj'
value, so that the name in the `ffi-obj' value can be used
by `_cprocedure'.
Closes PR 12645
The new predicates are `progress-evt?' `thread-cell-values?',
`prefab-key?', `semaphore-peek-evt?', and `channel-put-evt?'.
These were used internally, and now they appear in contract
error messages.
When supplying an accessor to redirect, either the corresponding field
must be accessible through the current inspector, or a mutator for
the same field must be redirected, too.
Stevie realized that we need this constraint; otherwise, impersonators
can implement mutator-like behavior even when the mutator is otherwise
secret.
Add `raise-argument-error', `raise-result-error', `raise-arguments-error',
and `raise-range-error'.
The old convention was designed for reporting on a single (sometimes very
long line). The new convention is
<name>: <short message>
<field>: <detail>
...
If <detail> is long or itself spans multiple lines, then it may
also use the form
<field>:
<detail>
where each line of <detail> is indented by 3 spaces.
Backtrace information is shown as a multi-line "context" field.
For example, if definitions have two unmarked `x's that originate
from different modules, make them correspond to different bindings.
This improvement will be used by `scribble/srcdoc', which will
rely on module context to connect `for-doc' requires to documentation
code that appears in the same module --- which is needed, for example,
if a macro expands to documentation code and the macro is used in
a different module.
These primitives atomically update a box to a new value, as long
as the current value is the same as a provided value. They also
are future-safe.
When futures are enabled, they use low-level hardware instructions
to perform the change atomically.
The expander no longer needs to generate certain phase-N
identifiers, since we now have a comparison operation that
uses different phases for each of two identifiers.
The preserved path is exposed by a new `module-path-index-submodule'
function, and `module-path-index-join' now accepts an optional
submodule path.
Also, fixed a problem with `collapse-module-path-index' when
a module path indx is built on a resolved module path that
is a submodule path.
In addition to the main repair, `collapse-module-path[-index]' is
correctly documented to allow '(quote <sym>) rel-to paths.
Finally, `collapse-module-path-index' changed to use a symbolic
resolved module path that appears as the base of a module path
index, rather than falling back to the given rel-to path. It's
possble that the old beavior was intentional, but it wasn't tested,
and it seems more likely to have been a bug.
Closes PR 12724
For example, `(module-declared? '(submod (planet dyoo/bf) reader) #t)'
shouldn't fail if there's no "main.rkt" to hold a `reader' submodule;
it should return #f.
Merge to 5.3, but updating cstartup.inc will require a manual merge.
AVL trees to tend to be shorter, which means a faster
search and insertion. The potential benefit of a red--black
tree's fewer rotations doesn't matter, I think, for a functional
variant, where you have to reconstruct a spine to the root,
anyway. The difference is small for typical tables, though it
can be as much as 50% for a large table with keys inserted in
order. And since the AVL code is also much simpler, why not?
The usual techniques: shortcut around generic scheme_apply() for
chaperone-triggered slow path, shortcut around scheme_apply() for
application of a native-code interposition function, and shortcut
`chaperone-of?' test by trying `eq?' first.
The immediate symptom was that `(provide (all-defined-out))'
didn't work in a `module+'-based submodule, but there were
also non-submodule ways to expose the problem.
Although th eoriginal idea was to distinguish "text" paths
from derived filesystem paths, practically everythign that accepts
a module path also accepts a path. Building the generalization into
`module-path?' makes it easier to support `submod' wrappers on paths,
and it seems to fix things rather than break them.
The bug happens with n-ary uses of arithmetic operations that
have constant arguments but couldn't be constant-folded ---
maybe due to a divide-by-zero.
It's possible for a deep recursion to be all in C instead of
JIT-generated code, in which case the caching code for
`current-continuation-mark' cannot kick in to make the operation
effectively constant time. Bail out (to keep things constant time) if
that happens.
The old implementation could cause deadlock by blocking on a semaphore
while waiting for the original place to run a callback, but a master
GC might be needed (and the blocked place wouldn't get the signal).
Beside fixing that problem, a potential memory leak is fixed in
calling an ffi funcition and having a Racket->C unmarshaling fail.
Also, the GC marking routine for a `place' value didn't reference the
place's underlying `place_obj' value.
The bug is triggered by unsafe flonum operations, a
conversion that tries to make the arguments more unboxable,
and a `lambda' form within an argument to the unsafe
operation.
Closes PR 12587
I now think the problem is likely to be realted to values
that do not fit into a signed 32-bit integer. Check for
the OS version and reject such integers.
The ActiveX part of MysterX is gone. The `ffi/com' re-imeplemtnation
provides only core COM support.
The "mysssink" DLL is still needed, and its source is still
in the tree, but it is downloaded in the same way as other
pre-built DLLs. The DLL no longer needs to be registered with
regsvr32.
The prohbition against `handle-evt' on `handle-evt' is as
document and as originally intended. I'm not sure why it
was allowed.
Existing programs that use `handle-evt' incorrectly
can break. I found and fixed one incorrect use and one
questionable use in the Racket tree (which is a small
minority of the uses of `handle-evt' in the tree).
Extend `define-cstruct' to support #:property specs, which causes
the constructor and C->Racket coercsions to wrap the pointer in
a structure instance with the specified properties. Of course,
the wrapper structure has a `prop:cpointer' property so that the
wrapper can be used transparently as a C pointer.
Add missing tests and documentation for the id`->list', `list->'id,
id`->list*', and `list*->'id bindings created by `define-cstruct'.
This addition triggered several other changes:
* -k for a Mac OS X embedding is now relative to the __PLTSCHEME
segment (which means that executables won't break if you strip
them, for example)
* the command-line no longer has a limited size for Mac OS X
launchers and embedding executables
* Mac OS X GUI and Windows launchers record the creation-time
collection path, unless they are created as "relative" launchers
In particular, allow a pair of a relative-to directory and a base
directory. Paths that syntactically extend the base directory are
recorded as relative to the relative-to directory (which must
syntactically extend the base directory).
The compilation manager now sets the parameter to a pair with
the base directory as the main collection directory, if the source
file's path extends that directory's path.
This generalization solves problems created by cross-module inlining,
where the source location of a procedure in bytecode can now be in a
different file than the enclosing module's file.
Also add a test that checks whether the build directory shows up
in any ".zo", ".dep", or documentation ".html" files.
Closes PR 12549
It looks like my bound for last time was too conservative,
in that I looked for the lowest number that didn't seem
to fail in 10.6. The range of failing values is apparently
not continuous.
I've tightened the bound to match the lowest
number that produces a useful result on my 10.7 machine,
assuming that it works for a continuous range there.
(The new bound is higher than the number previously used as
a lower bound.)
Merge to 5.2.1
Setting the environment variable causes the bytecode compiler to run
the bytecode validator (which is normally applied to input from a
bytecode file) immediately on all of the compiler's own results.
Certain `lambda'-lifting operations can cause information
about the flonumness of a variable to get lost, leading
to a mismatch between the closure's flags and flags on
a variable reference. (The bytecode validator could detect the
bug when loading the broken bytecode. The broken information,
meanwhile, was only used by the JIT.)
For numbers around -67768122973228093, localtime() doesn't return in
10.6.8, while it returns NULL for 10.7.2. Work around the bug by
setting a lower bound that seems to be high enough to avoid the
problem (and that's lower than the lowest value that succeeds, so no
results are lost, at least for now).
Merge to 5.2.1
If a function with a rest arg is called with argv not at the
start of the runstack, then space is allocated for the rest-arg list
on the runstack without clearing the allocated slot. The value in
the slot could be a pointer that wasn't traversed by the most recent
GC, so it could crash a GC during allocation of the rest-arg list.
Also, tweak setup code for a function of no arguments, and improve
comments in the code.
Merge to 5.2.1
The generated code was checking arity after potentially copying
arguments to the start of the runstack (i.e., if the arguments
were not already there). If too few arguments are provided, then
the copy might access past the end of the given array.
The redundant arity check removed in commit f7c506471b
had previously masked this problem. (Or the check wasn't redundant
in that sense, but it's better this way.)
Merge to 5.2.1
The problem is related to marks that should cancel eagerly when
a form passes through many layers of macro expansion, such as in
the sieve stress test for `syntax-rules'.
There's no particular reason that any one format will have all
the information that other formats need, but it conveniently works
for now that HTML info can subsume Latex info.
Certain unsafe operations were allowed to propagate across a
`lambda' boundary (where space safety is known not to be an issue),
which could lead to duplicate uses of a "once used" variable if
the relevant `lambda' is inlined.
Furthermore, `lambda' boundary crossing wasn't detected in the case
that the operation to propagate was propagated through an intermediate
variable without a `lambda' crossing.
Merge to 5.2.1
I'm fairly certain that the change in commit 25e9bd2a190acf861 isn't
right, but I'm having trouble generating tests to demonstrate the
original bug or this correction.
First use of the function was determining a single arity for
the enclosing module, and that arity could trigger warnings
in addition to failures to inline. For example, using `map'
on 3 arguments would trigger incorrect warnings for later
uses of `map' on 2 arguments.
Rename `read-intern-literal' to `datum-intern-literal'.
Interning is needed only in `read-syntax' or `datum->syntax' to
set up the invariants that the bytecode compiler needs for cross-module
optimization. When `read'ing numbers from a data file, meanwhile,
interning slows things down a lot and doesn't seem worthwhile.
Pass a pointer to the thread-local table on entry to JIT-generated
code, instead of having the JIT-generated code call a C function
to get the table. This doesn't seem to improve performance on my
machine, but it generates less code and is probably faster in
some cases.
When a future is blocked on JIT generation, a lightweight closure
is captured, and then the future moves on, the runtime thread would
correctly shift the on-demand JIT argument to the captured copy
of the runstack. However, it would also add 2 to that pointer
to use as the argv array, and the captured runstack is not allocated
to allow interior pointers, so a GC during on-demand JIT could
crash. The solution is to keep an offset alongside the argv pointer
during JITting.
The over-eager transformation could be space-unsafe, and it
could duplicate an unsafe operation whose result is used only
once in a function that eds up being inlined multiple times.
More generally, support a
(define _id (begin 'compiler-hint:cross-module-inline _proc-expr))
hint, which is how the compiler determines that `map', etc., are
candidates for inlining.
Inline only trivial functions, such as `(empty? x)' -> `(null? x)',
to avoid generating too much code.
Bytecode includes a new `inline-variant' form, which records a
version of a function that is suitable for cross-module inlining.
Mostly, the variant let the run-time system to retain a copy
of the bytecode while JITting (and dropping the bytecode of)
the main variant, but it may be different from the main variant
in other ways that make it better for inlining (such a less loop
unrolling).
