Commit 00d438cfbe made an attempt at this,
but this commit does it in a much more careful way, based on manually
emulating how the macroexpander expands module* forms in order to allow
splicing-syntax-parameterize to apply even within #%module-begin forms
introduced by the expander.
Normally, it's impossible to generate lots of `eq?`-hashing
collisions, but when the compiler inlines a function, it can duplicate
variables in a way that gives each copy the same `eq?` hash code. The
immutable-hash tree implementation failed when more than 32 collisions
occurred (which triggers a subtree in the collision node).
It's similarly very difficult to generate > 32 values that collide on
`eqv?` hashes but are not `eqv?` (although it must be possible using
exact rationals or complex numbers).
The rktio conversion lost the deregistration of file descriptors in an
internal fd-to-semaphore table building on kqueue/epoll, causing the
wrong semaphore to be checked for a later recycling of the file
descriptor. This bug mainly affects Linux and ports created by
`subprocess`, since kqueue is not used for pipes on Mac OS and BSD
variants. The bug does not affect network sockets (which are the
primary intended clients of epoll/kqueue support), since the relevant
semaphore is deregistered when a socket is closed.
Thanks to James Bornholt for discovering the problem and providing the
repair.
Closes#1769
Fix a problem with compile-time bindings added to a
namespace created by `module->namespace` for a module
that does not have a source file.
Possibly, there's a different fault that should be fixed that caused a
binding to use the module's instantiation-time module path index
instead of its compile-time module path index (which is what happens
when a file is involved). This repair fixes the problem in a general
way, though, and leaves further improvement to the reimplementation of
the expander in Racket (which already does not suffer from the bug).
Thanks to Alexis for providing the example.
In a non-tail position, a JIT-generated application of `apply`
retained the argument list until the called function returned.
Fix it to drop the reference to the list before the function
is called.
And unquoted-printing string contains a string to `display` in all
print modes. Although it could be implemented with a structure type
that has a printing function, `raise-arguments-error` further treats
unquoted-printing strings specially by not using the error value
conversion handler, so it reliably produces literal text in the error
message; that way, `raise-arguments-error` can be used to construct
more error messages.
The hack to implement `_union` without help from libffi failed when
the total size of the variants is too large. Try a different approach,
which involves a bet that the total size plus whether the content is
all floating-point numbers will be enough information for most cases.
Relevant to #1351
Add guard, if first argument to `find-relative-path` is not a path,
then don't call `path-convention-type` on it.
Thanks to Jack Firth for reporting:
https://github.com/racket/rackunit/pull/46
Avoid printing `(unquote @d)` or `(unsyntax @d)` as `,@d` or `#,@d`,
which would mean `(unquote-splicing @d)` or `(unsyntax-splicing @d)`
to the reader, by adding an extra space before the `@`.
Additionally this adds macros that distinguish between the chaperone redirects of prop-only vector
chaperones, function chaperones, and structure chaperones since each of these may store a vector
in the redirects field.
Auto fields were incorrectly recorded as immutable in a structure type
that is first generated from the prefab struct key instead of
`make-struct-type`.
Thanks to Deren Dohoda for the report.
In the process of extracting minimal Windows path encoding for rktio,
I noticed a decoding issue with a path that ends with an unpaired
high-surrogate value. Add a suitable tests and fix the old decoder
(although it will probably go away).
Closes#1721, which points out that the `for/list` expansion
introduced in commit 5e94a906cd interacts badly with a body
that captures a continuation plus Racket's current implemenation
of continuations.
When Racket one day gets a better implementation of continuations,
this change could be considered again, but the general question is
whether programs can detect or be affected by the size of the
continuation (when the programs don't directly control the
continuation creation --- otherwise continuation marks obviously
expose the size).
For a term
(lambda (arg-id ...) (define def-id _rhs) ... (arg-id def-id) ...)
the expander could take quadratic time in the number of `def-id`s
due to walking an environment to remove use-site scopes. (The
variant of the expander rewritten in Racket didn't have this
problem.)
This make-c-id allows an author to specify a convention for how
to connect and identifier defined with define-ffi-definer and
the actual symbol in the file.
* Adds docs.
* Adds tests.
* Adds history.
An authentic structure type is one whose instances cannot be
impersonated or chaperoned. The intended use of `prop:authentic` is to
annotate a library-private data structure where impersonators are
never needed internally for the data structure, and the declaration
lets the compiler produce less code and fewer branches by omitting
impersonator support.
When comparing a part of a hamt that is a collision node versus a
subtree node, a "hash code" was extracted from the collision node ---
but that's really a code for an integer key is that used for the
collision element. The comparison should instead use a code extracted
from the reference to the collision node (which is the code that is
common to all colliding keys).
Avoid the well-known possibility of quadratic handling of ephemeron
chains, where all ephemerons are immediately known, no keys are
immediately known, and each link in the chain has a value that refers
to the next link's key.
To aviod quadratic behior, attach a list of ephemerons to each page of
allocated objecst, where marking any object on the page triggers a
rescan of the ephemerons without waiting to rescan all ephemerons.
The test cases relied on arity and changing log to have an arity
of both 1 and 2 activated new (incorrect) tests. I fixed the
incorrect tests as they applied to the log function.
Make `log` in `racket/base` optionally accept a second argument.
The second argument is the log `base`. The docs also recommend
`fllogb` when precision is important.
* Error message when base is 1
* Added docs.
* Add tests.
Optimization to convert `(hash-ref <ht> <key> (lambda () <constant>))`
to `(hash-ref <ht> <key> <constant>)` didn't check that the `lambda`
for had zero argument.
Closes#1648
Fix problem with once-use tracking and delayed variable-use marking
that is performed for local function bodies. A delayed variable-use
registration might happen after a once-used variable is replaced by
its use.
This scenario is difficult to provoke, because the optimizer has to
first decide not to move a once-use function, and in a latter pass
decide to move it after all. There's not enough information to
retract the tentative use plus its transitive implications.
The solution is to avoid the generic once-use layer for `lambda` forms
whose uses are delayed (and that likely has a good effect on inlining
anyway). The other half of the solution is to avoid transitive use
marking on a once-used variable whose expression has been moved (and
there are no transitive things to skip, because that expression isn't
a `lambda` form).
When the second argument to `bytes-set!` is a reference to a
module-level variable that is definitely defined but not a known
constant, then an incorrect reordering was used that would cause
the third argument value to get overwritten before the call.
Closes#1601
The primitive `read` uses a shortcut --- a private "ungetc"
implementation --- that did not count position correctly for
non-ASCII characters.
Closes#1599
The documentation says that it should work on any output port,
although there's special treatment of ports that originate
from `pretty-print` itself.
Closes#1579.
After some reductions, the new rator advance less the effect
clocks than the original rator. For example in
(equal? x 7) ==> (eq? x 7)
(my-struct? x) ==> #t or #f
The lambdas can be marked as single valued and/or mark preserving.
With this information is possible to remove unnecessary wrapping
like the `values` in
(let ([f (lambda () '(1))])
(display f f)
(values (f)))
or in reductions like
(car (list (f))) ==> (values (f)) ==> (f)
Moreover, this is useful to test that the optimizer has marked
correctly the function f as single valued and mark preserving.
If a module has any sort of complex bindings, such as a definition of
a macor-introduced identifiers, then `module->namespace` and variants
(like `variable-reference->namespace`) need to recreate suitable
bindings. Make sure that the module-path index for recreated bindings
is the run-time one, not the compile-time one.
Closes#1584
To avoid moving expressions that may have a side effect, the optimizer must
recognize that in this position this will cause an error and advance
the virtual clock.
Currently the only primitive that is flagged as SCHEME_PRIM_IS_OMITABLE and
may have multiple return values is `values`.
Thanks to Robby for finding the original version of the test.
When a hash table or other special value appears immediately on the
right-hand side of `define-values`, it needs to be protected by an
explicit quote when writing to bytecode.
Closes#1580
Continuing the saga that includes 8190a7730d and d1ba9fbb6e, it turns
out that a 0-binding clause as the last one isn't so special after
all. A little later in the optimizer, now that we're sometimes moving
an error to the body, we can't assume that the body can be discard
if an error was detected.
Set up bindings and shift phases as needed to make
`variable-reference->namespace` work in a run-time position when the
enclosing module is instantiated at a phase other than 0.
Thanks to Rohin Shah for the bug report.
Support an external implementation of `read-syntax` by exposing
functionality that is currently internal to `read-syntax`: a srcloc
argument to a "special"-producing port function and wrapping special
results to reliably distinguish them from characters.
When multiple-binding `let-values` form is split into a single-binding
form on the grounds that the right-hand side will definitely error,
the optimizer's effect clocks were advance incorrectly.
Closes#1552
For example, an `unsafe-unbox` call should not be moved past the
call to an unknown function that might change a box's content.
Thanks to Sergey Pinaev for the report.
The objective of lookup_constant_proc and the first part of
optimize_for_inline was to find out if the value of an expression was a
procedure and get it to analyze its properties or try to inline it. Both
were called together in a few places, because each one had some special
cases that were missing in the other.
So, move the lookup and special cases from optimize_for_inline to
lookup_constant_proc, and keep only the code relevant to inlinig in
optimize_for_inline.
Both function have a similar purpose and implementation, so merge them to consider
all the special cases for both uses.
In particular, detect that:
(if x (error 'e) (void)) is single-valued
(with-continuation-mark <chaperone-key> <val> <omittable>) is not tail sensitive.
Also, as ensure_single_value was checking also that the expression was has not a
continuation mark in tail position, it added in some cases an unnecessary
wrapper. Now ensure_single_value checks only that the expression produces
a single vale and a new function ensure_single_value_noncm checks both
properties like the old function.
Adjust list and stream handling as sequences so that during the body
(for ([i (in-list l)])
....)
then `i` and its cons cell in `l` are not implicitly retained while
the body is evaluated. A `for .... in-stream` similarly avoids
retaining the stream whose head is being used in the loop body.
The `map`, `for-each`, `andmap`, and `ormap` functions are similarly
updated.
The `make-do-sequence` protocol allows an optional extra result so
that new sequence types could have the same properties. It's not clear
that using `make-do-sequence` is any more useful than creating the new
sequence as a stream, but it was easier to expose the new
functionality than to hide it.
Making this work required a repair to the optimizer, which would
incorrectly move an `if` expression in a way that could affect
space complexity, as well as a few repairs to the run-time system
(especially in the vicinity of the built-in `map`, which we should
just get rid of eventually, anyway).
Compile a `for[*]/list` form to behave more like `map` by `cons`ing
onto a recursive call, instead of accumulating a list to reverse.
This style of compilation requires a different strategy than before.
A form like
(for*/fold ([v 0]) ([i (in-range M)]
[j (in-range N)])
j)
compiles as nested loops, like
(let i-loop ([v 0] [i 0])
(if (unsafe-fx< i M)
(i-loop (let j-loop ([v v] [j 0])
(if (unsafe-fx< j N)
(j-loop (SEL v j) (unsafe-fx+ j 1))
v))
(unsafe-fx+ i 1))
v))
instead of mutually recursive loops, like
(let i-loop ([v 0] [i 0])
(if (unsafe-fx< i M)
(let j-loop ([v v] [j 0])
(if (unsafe-fx< j N)
(j-loop (SEL v j) (unsafe-fx+ j 1))
(i-loop v (unsafe-fx+ i 1))))
v))
The former runs slightly faster. It's difficult to say why, for
certain, but the reason may be that the JIT can generate more direct
jumps for self-recursion than mutual recursion. (In the case of mutual
recursion, the JIT has to generate one function or the other to get a
known address to jump to.)
Nested loops con't work for `for/list`, though, since each `cons`
needs to be wrapped around the whole continuation of the computation.
So, the `for` compiler adapts, depending on the initial form. (With a
base, CPS-like approach to support `for/list`, it's easy to use the
nested mode when it works by just not fully CPSing.)
Forms that use `#:break` or `#:final` use the mutual-recursion
approach, because `#:break` and #:final` are easier and faster that
way. Internallt, that simplies the imoplementation. Externally, a
`for` loop with `#:break` or `#:final` can be slightly faster than
before.
The optimizer can detect that some expressions will escape through
an error, and it can discard surrounding code in that case. It should
not change the tailness of a `with-continuation-mark` form by
liftng it out of a nested position, however. Doing so can eliminate
stack frames that should be visible via errotrace, for example.
This change fixes the optimizer to wrap an extra `(begin ... (void))`
around an expression if it's lifted out of a nested context and
might have a `with-continuation-mark` form in tail position.
In
(with-syntax ([x ....])
#'(x y))
and property on the source syntax object `(x y)` was lost in
constructing a new syntax object to substitute for `x`, while
properties on preserved literal syntax objects, such as `y`
were intact. Change `syntax` to preserve properties for
reconstructed parts of the template.
This change exposes a problem with 'transparent taint modes,
where the internal "is original?" property was preserved while
losing scopes that wuld cancel originalness. So, that's fixed
here, too.
extend optimize_ignore to go inside expressions with
begin, begin0 and let.
Also, try to reuse begin's in the first argument of
make_discarding_sequence.
If a mutable hash table changes while it's being printed,
various parts of the printing function could see a mismatch
between the current size and an old array size. To avoid this
problem, extract the size whenever extracting the array.
An identifier that gets a module context via `module->namespace` plus
`namespace-syntax-introduce` should not count as having the module as
its source as reported by `syntax-source-module`.
The correct behavior happened for the wrong reason prior to commit
cb6af9664c.
Closes#1515
For a template expression that involevs ellipses, a wrapper is added
to catch failures an report as an "incompatible ellipsis match count"
error. The wrapper was only added when there are multiple pattern
variables with ellipses, but it turns out that it's possible to fail
with incompatible counts using a single pattern variable.
Besides handlign that case, the revised check avoids an unnecessary
wrapper in cases where multiple pattern variables have ellipses but
they are used independently in a template.
Closes#1511
Without this repair,
#lang racket/base
(require 2htdp/abstraction)
(for/list ((dropping-which-one (in-naturals)))
1)
fails to compile with a "optimizer clock tracking has gone wrong"
error. A variant of this test (that doesn't depend on `2htdp`)
is now in the "optimize.rktl"; a simpler and more direct test
should be possible, but I wasn't able to construct one.
When a thread that is blocked on a set of semaphores and channels
is suspended and resumed after one of the events becomes ready,
and if the event has a wrapper function, then the wrapper was
not applied and the event selection was not reported correctly.
Thanks to Philip McGrath for reporting the problem.
This kind of reductions were applied only when x or y was a constant.
Classify the relevant predicates in 4 categories. In particular,
if <expr> satisfy pred? we can use this classification to apply
the correct reduction:
(equal? <expr> y) ==> [no reduction, unless y has a different type]
(equal? <expr> y) ==> (eqv? <expr> y)
(equal? <expr> y) ==> (eq? <expr> y)
(equal? <expr> y) ==> (begin <expr> (pred? y))
Also, add a new primitive interned-char? that is hidden, but it's
useful to track in the optimizer the the chars? with a value < 256
that are interned because they are treated specially, and if they
are equal? then they are eq?.
... + prefix-in + relative-path module. All of those ingredients
(or some similar alternatives) are necessary to trigger a slow
way of saving module context for interaction evaluation where
a module-path index shift was getting lost.
Previously the relevant predicates where disjoint, and until this commit
the only predicate that recognizes #f was `not`. So it's necessary to fix
two reductions to allow other predicates that recognize #f, like `boolean?`.
Add a hidden `true-object?` primitive that recognizes only #t, that is also
useful to calculate unions and complements with `boolean?` and `not`.
Also, extend a special case for expressions like
(or (symbol? x) (something))
where the optimizer is confused by the temporal variable that saves the
result of `(symbol? x)`, and the final expression is equivalent to
(let ([temp (symbol? x)])
(if temp #t (something)))
This extension detects that the temporal variable is a `boolean?` and
reduces the expression to
(if (symbol? x) #t (something))
* Wrong contract for syntax-local-value in the documentation.
* Clarified signature in documentation for expand-import, expand-export and pre-expand-export
* Corrected typo in documentation for "for".
* Fixed error message for function which seems to have been renamed in the docs
* Fixed typo in a comment in the tests
* Fixed a typo in the documentation for set-subtract.
* Use double ellipses for the free-id-table-set*, free-id-table-set*!, bound-id-table-set* and bound-id-table-set*! operations
The optimizer assumed a fixnum result if either argument to
`bitwise-and` implies a fixnum result. That's not correct if the
fixnum agument is negative.
Thanks to Peter Samarin for a bug report.
Merge to v6.7
When calling a procedure that is attached as a
`prop:rename-transformer` property value, make sure that
any available expansion context is accessible as reflected by
`(syntax-transforming?)`.
Syntax parameters as rename transformers particularly rely on that
information for local expansion.
Thanks to Jay for the "stxparam.rktl" test.
Closes#1479
In a pattern like
a*b
a naive attempt to match will take quadratic time on an input that
contains all "a"s an no "b". To improve that case, the regexp compiler
detects that a match will require a "b" and checks the input for a "b"
to enable linear-time failure.
That optimization mishandled `(?!...)` and `(?<!...)` patterns,
treating the must-not-match subpatterns as things that must match.
So,
(regexp-match "a*(?!b)" "aaaxy")
returned false, because the input doesn't contain "b".
Thie commit repairs the optimization.
Closes#1468
Fix a regression relative to v6.4 caused by a refactoring of the
compiler between v6.4 and v6.5. The refactoring lost information about
letrecs that are converted internally to let* when a mutable variable
is involved, and it ends up allocating a closure before the box of a
mutable variable that is referenced by the closure. Something like
`with-continuation-mark` is needed around the closure's `lambda` to
prevent other optimizations from hiding the bug.
Closes#1462
The `if` case of the compiler's space-safety pass abused its "last
non-tail call relative to the closest enclosing binding" state as
"last non-tail call relative to the enclosing run time", which could
cause it to not clear a stack position as needed to maintain space
safety.
This makes two changes to the forms in racket/splicing to adjust how
syntax properties are propagated through expansion:
1. Uses of make-syntax-introducer are passed #t as the first argument,
which causes the introduced scope to be consider a use-site rather
than macro-introduction scope. This prevents syntax objects from
being unnecessarily marked as unoriginal, in the syntax-original?
sense.
2. Uses of syntax/loc have been adjusted to copy syntax properties
from original syntax objects, which were previously discared. Forms
that were spliced into the surrounding context, such as begin,
define-values, and define-syntaxes, recreated the top-level syntax
objects, which did not preserve syntax properties from the
originals.
This is not a perfect solution, mostly because it potentially elides
properties that may be associated with captured literals (that is,
properties attached directly to begin, define-values, or define-syntaxes
identifiers themselves). However, it seems to accommodate most of the
common use-cases: propagation of syntax-original?-ness and forms like
`struct`, which attach properties like 'sub-range-binders.
fixes#1410
A `struct-copy` form can generates a call for a constructor that
includes a sequence of `unsafe-struct-ref` arguments. Each
`unsafe-struct-ref` must still check for a chaperone. Make the JIT
recognize that pattern an turn it into a single test instead of
one test per `unsafe-struct-ref`.
Make the optimizer recognize and track `make-struct-property-type`
values, and use that information to recognize `make-struct-type`
calls that will defnitely succeed because a property that hs no
guard is given a value in the list of properties.
Combined with the change to require-keyword expansion, this
change allows the optimizer to inline `f` in
(define (g y)
(f #:x y))
(define (f #:x x)
(list x))
because the `make-struct-type` that appears between `g` and `f`
is determined to have no side-effect that would prevent `f` from
having its expected value.
When a module defines and exports an identifier at two phases,
and when another module imports both of them at the same phase,
an error was not reported as it should have been.
