Now accepts any whitespace, not just spaces, ignores both leading and
trailing whitespace, and accepts multiple whitespace characters
separating subterms.
After refactoring the test for the inferred types of some procedures that
use vector?/bytes?/string?/list? it was easier to spot the missing information.
Note that in the documentation, some arguments like the position in
(vector-ref <vector> <position>)
are documented as exact-nonnegative-integer? but due to the implementation
details they are actually in a subset of fixnum?s.
When `read` parses a literal hash table, it inserts an placeholder
just in case it's needed for cycles. The `unsafe-immutable-hash-...`
operations in some cases did not detect and remove the placeholder.
Closes#1376
Merge to v6.6
The namespace returned by `variable-reference->namespace` (or
`namespace-anchor->namespace`) may be used via `eval` to define new
bindings, so enable top-level binding support for the namespace.
for example, make the optimizer convert something like
(struct a (x))
(lambda (v) (if (a? v) (a-x v) #f))
to
(struct a (x))
(lambda (v) (if (a? v) (unsafe-struct-ref v 0) #f))
The optimizer change in e887fa56d1 recognized struct declarations that
involved only whitelisted properties to guarantee that constructor
properties are preserved --- while `prop:chaperone-unsafe-undefined`
can affect the constructor, and other properties might imply that one.
But the optimizer's transformer aren't actually invalidated by
`prop:chaperone-unsafe-undefined`; the JIT's assumptions are affected,
but that's handled in a different way. So, remove the whitelist and
allow any property list.
This change permits EH patterns like (~once <nullable>), because the
match will happen at most once, so there is no danger of divergence.
Thanks to Alex Knauth for pointing out this special case.
Merge to release branch.
The optimizer tries to reduce the `if` assuming that the result will be used.
In case it later detects that the result will be ignored, it can try to
apply some additional reductions to the branches and to the whole expression.
This function exposes the fast subset operation that is built in for
immutable hash tables (and used by the set-of-scopes implementation).
Also, make the space optimization implicit for `eq?`-based hash tables
that contain only #t values (instead of explicit and only available
internally). It turns out to be easy and efficient to make the
representation automatic, because the HAMT implementation can support
a mixture of nodes with some containing explicit values and some
containing implicit #t values.
Now with-disappeared-uses surrounds its body with let, so it can contain
multiple body expressions. The record-disappeared-use function is like
record-disappeared-uses but for a single identifier.
The `#%linklet` module is intended to eventually provide
a simplified compiler for the core Racket language. For
now, it provides minimal hooks for bootstrapping an
expander implementation.
When the properties argument for `make-struct-type` is non-empty,
then we cant; guarantee that `make-struct-type` succeeds, but
if it does, then we can still know that the result is a structure
type and (as long as `prop:chaperone-unsafe-undefined` is not
involved) the properties don't affect the constructor, predicate,
selector, or mutators.
The resolve-module-path-* functions effectively already had a default argument,
which is #f, this allows you to just directly call it with one argument.
This fixes an immediate problem, but the macro expander should have
complained about an unbound `maybe` at phase 2. (A new implementation
of the macro expander detected the unbound `maybe`.)
In (~and p1 p2), a failure in p2 now always dominates a failure in p1.
Consequently, if a pattern succeeds, its failures don't matter.
Add {pat,hpat,action}:ord wrappers, ord prframes. Apply ordering to
main pattern and side clauses. Add better progress analysis to
eliminate order wrapping.
When an array value is provided, make sure that it's an array
with at least the expected length (or longer) and same element
layout. That's weaker than checking that the array elements have
the right type, because an `eq?` check at the ctype layer seems
too strong, and the ctype API doesn't provide enough information
for a more flexible equality.
Avoid creating a result that is intended as a module path but
has elements that are not syntactically allowed, such as a "."
in a collection-path element.
A phase shift was mising on `begin-for-syntax`es introduced by
`syntax-local-lift-module-end-declaration`, which is in turn
used to implement` module+`, so `module+` didn't work under
two or more `begin-for-syntaxes`.
Closes#1312
Syntax objects generally make sense as properties in other syntax
objects, but they require special care when marshaling to bytecode
(as syntax objects do in general). To make that special handling
possible and reliable, constrain the shape of allowed values.
The name `path-extension` created a conflict for an existing
registered package, so it should not have been added to
`racket/path`.
Also, `path-get-extension` was intended to work on a path
that is syntactically a directory, so fix and test that.
Change the one expansion mode as far as I can tell) that disables
lifts so that lifts are now allowed, which means that
`(syntax-transforming?)` implies `(syntax-transforming--with-lifts?)`.
The old documentation incorrectly characterized when lifts
were allowed. Ryan noticed the documentation problem, and that
observation led to this simplication.
This is a partial solution to the ~and problem, only for side clauses.
In (~and p1 p2 p3), one often wants errors in p2 to take precedence over
errors in p1, and likewise for p3 over p2. One solution is ~commit, but
that prevents backtracking. Another is ~post, but then two ~post wrappers
are needed around p3. Also, it doesn't make sense to compare progress of
the third #:with clause from stxclass A to the second #:with clause of
stxclass B and say third beats second.
So, generalize 'post to (post group index); post frames are comparable to
each other only if group is the same, then compared by index. (Post still
beats CAR and CDR.) Each set of side clauses shares a group.
For simplicity of code generation for now, use gensyms to identify groups.
A `#:name` identifier picks the name that is bound to static
information about a structure type. An `#:extra-name` identifier
specifies an additional name to be bound to the information.
This pair of options is analogous to `#:constructor-name`
and `#:extra-constructor-name`.
Based on Jen Axel's suggestion and implementation.
Closes#1309
Provide a cleaned-up set up path-extension functions. In contrast
to `path-{add,replace}-suffix` and `filename-extension`, a dot
at the beginning of a path element is not treated as an extension
separator. Also, `path-extension` returns an extension including
its separator, which is more consistent with other extension
functions.
The new `path-has-extension?` function replaces many uses of
regexp matching in the base collections.
Closes#1307
Adjusts the docs, but a future replacement (possibly with a new name
to avoid compatibility issues) should behave like the old docs,
instead.
Closes#1292
restricted through `get-info`, which prunes the environment
variable set before it loads the "info.rkt" file. All
environment variables are pruned except those listed in
`PLT_INFO_ALLOW_VARS` (separated by semicolons).
Related to emina/rosette#17.
Add a hook to `raco setup` to make copies of installed executables,
where the copies start with the configuration or addon directory
of creation time, instead of the default installation or user-specific
path.
Although the same effect can be achived by setting environment
variables such as PLTADDONDIR, tethered executables can be easier
to work with and compose better with other programs.
See also #1206 for some discussion, although this change does
not exactly address the original idea there.
Reduce (unbox (box x)) => x
Extend the reductions for cXr to the unsafe versions, for
example reduce (unsafe-car (cons x y)) => x
Check and save types in unsafe operations
Pass a string to the handler to describe the problem.
Also, fix minor issues (GC registration, contracts and `history`
in docs) and make `pregexp`, etc., report compilation errors as
`pregexp`, etc.
Allow `system-type` on non-Windows platforms to run `uname` to get
machine information, even in a sandbox or other contexts with a
limiting secutiry guard.
Check that it works to apply a continuation that shares with
an enclosing continuation, where a runstack overflow happens
between the continuations.
Closes PR 15281
While expanding a module, the root of module-relative references is a
fresh notion of "this module".
After expansion, "this module" is shifted to "an expanded module",
which is a global constant (for top-level modules). When an expanded
module is re-expanded, "an expanded module" is shifted to a fresh
"this module" during re-expansion, and so on.
One problem with this approach is that the shift from "this module" to
"an expanded module" isn't applied to syntax properties --- but
there's some extra trickery to make it work out by mutating "this
module" to make it look like "an expanded module".
Submodule expansion introduces an intermediate "parent of this module"
that wasn't currently covered by the extra trickery, so fix that.
`->i` already supports method contracts (for use wihin `object-contract`,
whose `->i` support is tested, but undocumented), which would make `->im`
possible.
Unfortunately, that support is very incomplete, missing support for using
`this` in contracts, making this `->im` (or the undocumented `->i` +
`object-contract` combo) basically useless.
Once/if that is added, then this commit would enable `->im`. Until then,
it's mostly useful for future reference (hence will be reverted).
In the meantime, it's possible to use `->i` within class/object contracts
with an explicit `this` argument, so nothing critical is lost, just a tiny
shortcut.
Previously, on the example, the function was not anonymous, so no
`tempN' would appear on the error message.
The fixed example makes use of `flat-named-contract' on an anonymous
function, which resembles the snippet above it and demonstrates the
purpose of `flat-named-contract' better.
Repair a mismatch between `syntax-local-lift-expression` and the
way that `compile` tries to avoid creating bindings while
compiling a top-level `define` form.
Closes#1284 and #1282
Some functions were passing when they shouldn't have, only to fail when
the function is called.
Technically not backwards compatible, but should only affect functions
that were never called.
A syntax property is added as preserved or not. For backward
compatibility, the default for a 'paren-shape key is preserved, and
any other key's default is non-preserved.
Cross-module inlining that pulls a variable reference across a
module boundary imposes a more struct requirement that run-time
"constant" detection is consistent with the optimizer's view of
"constant" within a module. So, make sure they're the same.
Sometimes the optimizer removes all the references to a variable but it
doesn't detect that the variable is unused, so it keeps the definition.
Later, the sfs detects the unused variable so it marks it, but it doesn't
remove the let form.
Using `syntax` to capture local binding information in the current
phase doesn't work with the set-of-scopes expander. Although the
example could be adjust to use `(quote-syntax car #:local)`, it
seems like too much detail at that point in the explanation.
Formerly, cross-module inlining would not work for a function like
(define (f x)
(if .... .... (slow x)))
unless `slow` was also inlined into `f`. This commit changes
cross-module inlining so that it allows a call to `f` to be replaced
with an expression that references other module-level bindings (that
are not primitives), such as `slow`.
Adjusting the inlining rules can always make some program worse. In
this case, a hueristic about whether to export an optimized or
unoptimized variant of a fnuciton for inlining tends to collide with
the adjusted inlining rule, so this commit tweaks that heuristic, too.
Enable the optimizer to figure to figure out that a loop
argument is always a real number, for example, in much the
same way that it can detect fixnums and flonums for unboxing.
Unboxing information was only needed at the resolve level,
but `real?` information is useful only to the optimizer, so
the generalization enables the optimizer to reach
approximations of type information earlier (e.g., among
a subset of a function's arguments).
Simplify `(wcm <k1> <v1> (wcm <k1> <v2> <e>))` to
`(begin <v1> (wcm <k1> <v2> <e>))` for a simple enough <k1>.
A variable simple enough, so this is useful for improving
errortrace output.
Compute an `equal?` hash code for `read`able values that
is a constant, at least for a given version of Racket. Only
(interned) symbols failed to have that property before.
An `XFORM_NONGCING_NONALIASING` function doesn't trigger a GC, and
when it is given an argument that is an address of a local variable,
it fills in that address and doesn't leak it. This annotation allows
the xform transformation (to support precise GC) avoid some work for
some hash-iteration functions.
With the old representation of local variables, optimize_info_lookup
had to search the stack for the frame with the information about the
variable. This was complicated so it has many flags to be used in
different situations and extract different kind of information.
With the new representation this process is easier, so it's possible
to split the function into a few smaller functions with an easier
control flow.
In particular, this is useful to avoid marking a variable as used
inside a lambda when the reference in immediately reduced to a
constant using the type information.
The iterator saves the return points in a list. For small immutable hashes,
encode the values in the list in the bits of a fixnum to avoid allocations.
Expose tagged allocation and a function that interprets a description
of tagged shapes. As a furst cut, the description can only specify
constant offsets for pointers within the object, but future extensions
are possible.
When a chaperone-wrapped function leads to a slow-path tail
call, the continuation-mark depth can be made too deep when
resolving the slow tail call.
Closes#1265
Reduce
(eq? v v) ==> #t
(if t v v) ==> (begin t v)
(if v v #f) ==> v
when v is a local or a top level variable.
Previously, the last two reductions were used only
with local variables.
Also, move the (if x #t #f) ==> (not x) reduction
after branch optimization.
When a key is removed at a level that other only has a collision
table, the HAMT representation was not adjusted properly by
eliminating the layer. As aresult, table comparison via
`equal?` could fail. The problem could show up with hash tables
used to represent scope sets, where an internal "subset?" test
could fail and produce an incorrect binding resolution.
The transformation from
(begin (let <bindings> (begin <e1> ...)) <e2> ...)
to
(let <bindings> (begin <e1> ... <e2> ...))
makes things look simpler and might help the optimizer a little. But
it also tends to make the run-time stack deeper, and that slows some
programs a small but measurable amount.
A better solution would be to keep the transformation but add another
pass that moves expressions out of a `let`.
Since this operation only moves the code and doesn't make the final
bytecode bigger, it's not necessary to decrease the fuel and then it
is available for further inlining.
The calculation of used variables in a possibly unused function did
not work right when the function is referenced by a more deeply
nested function that itself is unused. The extra uses triggered by
more nested uses need to be registered as tentative in the more nested
frame, not in the outer frame.
Closes#1247
On Unix and OS X, the check to avoid replacing an existing
file or directory is made by Racket, rather than the OS,
so don't claim a system error if the operation fails for
that reason.
Also, update the docs to clarify that the check is not
atomic with the move.
Closes issue #1158
Correct the second-biggest design flaw in the bytecode optimizer:
instead of using a de Bruijn-like representation of variable
references in the optimizer pass, use variable objects.
This change is intended to address limitations on programs like the
one in
http://bugs.racket-lang.org/query/?cmd=view&pr=15244
where the optimizer could not perform a straightforward-seeming
transformation due to the constraints of its representation.
Besides handling the bug-report example better, there are other minor
optimization improvements as a side effect of refactoring the code. To
simplify the optimizer's implementation (e.g., eliminate code that I
didn't want to convert) and also preserve success for optimizer tests,
the optimizer ended up getting a little better at flattening and
eliminating `let` forms and `begin`--`let` combinations.
Overall, the optimizer tests in "optimize.rktl" pass, which helps
ensure that no optimizations were lost. I had to modify just a few
tests:
* The test at line 2139 didn't actually check against reordering as
intended, but was instead checking that the bug-report limitation
was intact (and now it's not).
* The tests around 3095 got extra `p` references, because the
optimizer is now able to eliminate an unused `let` around the
second case, but it still doesn't discover the unusedness of `p` in
the first case soon enough to eliminate the `let`. The extra
references prevent eliminating the `let` in both case, since that's
not the point of the tests.
Thanks to Gustavo for taking a close look at the changes.
LocalWords: pkgs rkt
Found with `-fsanitize=undefined`. The only changes that are potentially
bug repairs involve some abuses of pointers that can end up misaligned
(which is not an x86 issue, but might be on other platforms). Most of
the changes involve casting a signed integer to unsigned, which
effectively requests the usual two's complement behavior.
Some undefined behavior still present:
* floating-point operations that can divide by zero or coercions
from `double` to `float` that can fail;
* offset calculations such as `&SCHEME_CDR((Scheme_Object *)0x0)`,
which are supposed to be written with `offsetof`, but using
a NULL address composes better with macros.
* unaligned operations in the JIT for x86 (which are ok, because
they're platform-specific).
Hints for using `-fsanitize=undefined`:
* Add `-fsanitize=undefined` to both CPPFLAGS and LDFLAGS
* Add `-fno-sanitize=alignment -fno-sanitize=null` to CPPFLAGS to
disable those checks.
* Add `-DSTACK_SAFETY_MARGIN=200000` to CPPFLAGS to avoid stack
overflow due to large frames.
* Use `--enable-noopt` so that the JIT compiles.
The `alarm-evt` tests are inherently racy, since they depend on
the scheduler polling quickly enough. The old time values were
close enough that a test failure is particularly likely on
Windows, where the clock resolution is around 16ms. To reduce
failures, make the time differents much bigger.
Closes issue #1232
A reference to a local may be reduced in a branch to a constant, while it's unchanged in the
other because the optimizer has different type information for each branch. Try to use the
type information of the other branch to see if both branches are actually equivalent.
For example, (if (null? x) x x) is first reduced to (if (null? x) null x) using the type
information of the #t branch. But both branches are equivalent so they can be
reduced to (begin (null? x) x) and then to just x.
The functions expr_implies_predicate was very similar to
expr_produces_local_type, and slighty more general.
Merging them, is possible to use the type information
is expressions where the optimizer used only the
local types that were visible at the definition.
For example, this is useful in this expression to
transform bitwise-xor to it's unsafe version.
(lambda (x)
(when (fixnum? x)
(bitwise-xor x #xff)))
Skip calling the domain projection in that case and, if all of the
arguments are any/c then also skip putting the contract continuation mark
This appears to give about a 20% speed up on this program:
#lang racket/base
(require racket/contract/base)
(define f
(contract
(-> any/c integer?)
(λ (x) 1)
'pos 'neg))
(time
(for ([x (in-range 4000000)])
(f 1)))
The recently added fast path for property-only chaperones did not
propagate the original object in the case that the property-only
chaperone wraps a `chaperone-procedure*` chaprerone.
Merge to v6.4
Fix `procedure-specialize` for a procedure that refers to a
syntax-object literal. A syntax-object literal becomes part of the
procedure closure, but in a special way that nomrally allows syntax
objects to be loaded on demand. For now, specialization counts as
a demand of the syntax object.
Merge to v6.4
When a procedure created by `unsafe-{chaperone,impersonate}-procedure`
is given the wrong number of arguments, the original procedure's name
should be used in the error message.
During inlining, the type information gathered in
code that was inside the lambda is copied to the outer
context. But the coordinates of the type information
were shifted in the wrong direction, so the type was
assigned to the wrong variable.
This bug is difficult to trigger, so the test is convoluted.
Merge to v6.4
Add 'module-body-inside-context, 'module-body-outside-context, and
'module-body-context-simple? properties to the expansion of a
`module` form. These properties expose scopes that are used by
`module->namespace` and taht appear in marshaled bytecode.
The expression in a `define-runtime-path` form is used in
both a run-time context and a compile-time context. The
latter is used for `raco exe`. In a cross-build context,
you might need to load OpenSSL support for Linux (say)
at build time while generating executables that refer to
Windows (say) OpenSSL support. In that case, `#:runtime?-id`
lets you choose between `(cross-system-type)` and
`(system-type)`.
Merge to v6.4
