Progress toward making the bytecode compiler deterministic, so that a
fresh `make base` always produces exactly the same bytecode from the
same sources. Most changes involve avoiding hash-table order
dependencies and adjusting scope identity. The namespace used to load
a reader extension is also better defined. Plus many other little
changes.
The identity of a scope that is unmarshaled from a bytecode file now
incorporates the hash of the file, and the relative order of scopes is
preserved in a bytecode file. This combination allows compilation to
start with modules that loaded and compiled in different orders
(including delayed loading of bytecode fragments within one file).
Formerly, a reader extension triggered by `#lang` or `#reader` was
loaded in whatever namespace happens to be current. That's
unpredictable and can pollute a module build at the level of bytecode.
To help make builds deterministic, reader extensions are now loaded in
a root namespace of the current namespace.
Deterministic compilation in general relies on deterministic macros.
The two most common ways for a macro to be non-deterministic are by
using `gensym` (use `generate-temporaries`, instead) and by using an
unsorted hash-table traversal (don't do that).
At this point, bytecode generation is unlikely to be completely
deterministic, since I uncovered non-determinism mostly by iterating
attempts over the base collections. For now, the intent is not to
provide guarantees outside of the compilation of the base collections
--- but "more deterministic" is likely to be useful in the short run,
and we can improve further in the long run.
Specialize a
(call-with-immediate-continuation-mark _key (lambda (_arg) _body) _def-val)
call to an internal
(with-immediate-continuation-mark [_arg (#%immediate _key _def_val)] _body)
form, which avoids a closure allocation and more.
This optimization is useful for contracts, which use
`call-with-immediate-continuation-mark` to avoid redundant
contract checks.
On OS X, it seems that access() can sometimes fail with EPERM
when checking for execute permission on a file without it.
I've previously seen this result when running as the superuser,
but that's apparently not the only possibility; a long path
may also be relevant.
Re-linking in a new namespace doesn't need the namespace of
compilation.
A "namespac.rktl" test exposed this problem, where the "transfer a
definition of a macro-introduced variable" test could fail if a GC
occurred between compilation in one namespace and evaluation in
another.
Although `eval-syntax` is not supposed to add the current namespace's
"outer edge" scope, it must add the "inner edge" scope to be consistent
with adding the inner edge to every intermediate expansion (as in
other definition contexts).
In addition, `eval`, `eval-syntax`, `expand`, and `expand-syntax`
did not cooperate properly with `local-expand` on the inner edge.
Some failure paths were missing an update before calling failure
code, and the new failure paths need to unconditionally update the
runstack pointer (because the common stub doesn't know whether the
calling context needs an update).
Genereating a use-site scope, instead of a macro-introduction scope,
prevents the scope's presense from triggering a #f result from
`syntax-original?`.
This change mostly reverts 1465ff25fc, which turned out to be a hassle
because it created more cyclic structure.
A simpler strategy is to allow a phase-specific scope to be detached
(perhaps temporarily, due to on-demand loading of bytecode) from its
group; when that's possible, the scope is not reachable from a place
where it can be moved to other syntax objects, so it's ok to be
detached. Debugging output needs to handle that gracefully, though.
Also, in case of broken bytecode, fix up a detached scope if it
does end up in an unexpected place.
Formerly, compiling a definition in one namespace and evaluating it in
another would cause the definition to take place in the original
namespace --- unless the compiled code is marshaled to a byte string
and back. Adjust the "linking" process to redirect the variable
definition and any references to the new namespace. (This is a change
relative to the compiler with the old macro expander.)
Also, repair a compiled `require` form along similar lines. (This is
*not* a change relative to the compiler with the old macro expander;
the mismatch is part of the motivation for changing `define`
handling.)
Add the current definition context's scope to any expression that is
produced by macro expansion before trying to expand again, in case the
expansion needs to refer to a definition introduced by a previous
expansion.
Previously, the scope was added before any expansion and after any
expansion, but that misses intermediate points.
The old expander had this bug, too (some of the new tests fail there),
but it showed up less often and was sometimes considered correct, for
various reasons.
I had tried to simplify the "generation 0" allocation function to
always use `GEN0_PAGE_SIZE`, but "generation 0" is also used for place
messages, in which case a much smaller size should be used.
The "place-in-channel-fnl.rkt" test exposed this problem.
A recent GC change (included with the set-of-scopes expander)
allows the GCs marking procedure to recur directly to a limited
depth, instead of always pushing pointers onto a stack. Direct
recursion is not cmopatible with ephemeron-resolution process,
so switch to no-recur mode.
This problem was uncovered by an existing test.
The combination of splitting a `letrec` and optimizing
the resulting `(let ([x <proc>]) x)` to just `<proc>`
used a bad coordinate shift, which made property testing
incorrect, etc.
For reasons that are not clear, the new expander triggered
the problem through an existing test.
The `eval-syntax` function (which is used by other functions, such as
loading a module) should not install fallback-binding scopes from
the current namespace.
When `(let ([x ...]) (let ([y x]) ... y ... y ...))` turns into
`(let ([x ...]) ... x ... x ...)`, make sure that `x` is not
still marked as single-use. Incorrect marking as single-use could
cause the optimizer to inline too much, for example.
Thanks to Gustavo for tracking down the problem.
Previously all the predicates recognized only non-#f things, so ´not´ can be
added to the list of disjoint predicates. But many of the parts of the code
relied on the non-#f property and had to be modified.
In (if (eq? x <pred?-expr>) <tbranch> <fbranch>) infer that the type of
x is pred? in the tbranch.
Also, reduce (eq? x y) => #f when the types are different.
The optimizer reduces the variables with a known type to #t in a Boolean context.
But some predicates imply that the variable has a definite values, so they can be
reduced in a non-Boolean context too.
For example, in (lambda (x) (if (null? x) x 0))) reduce the last x ==> null.
This fixes the bug twice:
* Don't reduce mutable variables with a type to #t in a Boolean context.
* Don't record the type of mutable variables when a predicate is
checked in a test condition.
While reducing some ignored constructors, the optimizer may wrap the arguments
<expr> in (values <expr>) to ensure that it's a single value non-cm expression.
This avoids the unnecessary nesting of (values (values <expr>)).
Similarly, add the cases for begin and begin0 to single_valued_noncm_expression
While `#:in-original-place? #t` provides one way to serialize
foreign calls, it acts as a single lock and requires expensive
context switches. Using an explicit lock can be more efficient
for serializing calls across different places.
For example, running "plot.scrbl" takes 70 seconds on my machine
in the original place and using `#:lock-name` in any place,
while it took 162 seconds in a non-main place with Cairo+Pango
serialization via `#:in-original-place? #t`.
Internally, the named lock combines compare-and-swap with a
place channel. That strategy gives good performance in the case
of no contention, and it cooperates properly with the Racket
scheduler where there is contention.
The optimizer was able to use the type information gained outside
the let's to reduce expressions inside the lets. For example, in
(lambda (z) (car z) (let ([o (random)]) (pair? z)))
it reduces (pair? z) ==> #t.
This enable the propagation in the other direction so in
(lambda (z) (let ([o (random)]) (car z)) (pair? z))
it reduces (pair? z) ==> #t too.
Using `(thread-resume t1 t2)` would not prevent a GC of t1, but it
would create an intermediate record to make the link from t1 to t2,
and that intermediate record would leak due to a missing level of
indirection in a table-cleanup traveral. The leak not only accumulated
memory, it also caused ever slower traversals of the table in an
attempt to clean up.
(Since the leak is small and the leaking object is not directly
accessible, I don't have a good idea on how to test this repair
automatically, but see the program in the PR.)
Closes PR 15099.
Modern OS configurations likely use an even larger buffer size, and
making it small can have substantial negative performance effects
(e.g., with PostgreSQL over TCP).
When AC_PROG_CC picks GCC, move its selection of CFLAGS
into CPPFLAGS, so that preprocessing will have the same
optimization and debugging flags as compilation.
Arguably, AC_PROG_CC plus AC_PROG_CPP should do that
soemhow, but it's understandable that the autoconf
implementers didn't cover the possibility of
preprocessing that changes with the optimization level.
Closes#945
When `local-require` is used in a non-phase-0 position and it is
`expand`ed (as opposed to compiled directly), then the generated
`#%require` form had the wrong binding phase.
Merge to v6.2
In many use cases the length of the vector is fixed and know,
so we are sure that make-vector will not raise an error and
we can recognize these expressions as omittable and drop
them when the result is ignored.
The result of some procedures is a vector, but they are not omittable
because they may rise an error. With the recent changes of the
predicate reduction these cases are correctly handled.
The optimizer checks the type of the argument of some unary procedures and
uses the gathered information to replace them by the unsafe version, reduce
predicates and detect type errors. This extends the checks to more procedures
that have no unsafe version and procedures that have more than one argument.
Use the given readtable more consistently to parse
delimiters in the top-level form. This change particularly
addresses problems with trying to restore the original
`(` when parsing a hash table, but allowing nested
forms to still use a different `(` mapping.
When determing whether expressions can be reordered, a reference to a
module-defined variable was considered unreorderable when it is
known to have a value and no further mutation, but the value isn't
constant across all runs.
The optimizer had some reductions of predicates applications, like (pair? X),
only when X was very simple and the type of X was obvious.
Use expr_implies_predicate and make_discarding_sequence to allow
the reduction of more complex expressions.
Also, the reduction of procedure? and fixnum? were special cases in
optimize_application2. Move the checks to expr_implies_predicate
to take advantage of the reductions in more general cases.
Use `syntax-track-origin` and 'disappeared-use properties to
communicate `require` and `provide` form bindings to tools such as
Check Syntax.
Relevant to PR 13186
When a structure type has `prop:inpersonator-of`, follow it
when attemptng to access imperonator properties.
This change fixes a problem with `impersonate-procedure` as
reported by Scott Moore.
The compiler/expander attempted to clear out references in a namespace
used only during macro expansion, but it's possible for references to
be retained (via unusual macros), so get rid of the broken attempt to
help the GC.
Gustavo's tests in de3fa9a855 illustrate the problem. The solution
is simply passing 1 for `optimized_rator` to optimize_for_inline().
Additional changes generalize optimize_for_inline() a little (although
that generality doesn't seem to be useful at the moment) and collapse
some variables that represent the same value.
A new `--enable-ios=<sdk-path>` flag in combination with `--host=...`
sets up the right compiler options for compiling the Racket runtime
system as a framework to use in an iOS application.
I don't know whether the resulting framework actually works, but
compiling and linking is a step forward.
scheme_optimize_apply_values reduces (call-with-values gen proc)
to (#%apply-values proc gen) when recognizes proc as a procedure.
This extends the expressions that are recognized as procedures.
Instead of delaying the registration of some constants until a
group of expressions is re-optimized, add constant information as
it is discovered, which can expose some additional optimizations.
The old grouping was probably aimed at avoiding excessive code growth,
but I think that other and better controls are now in place. The
overall size of ".zo" files in an installation did not grow
significantly with this change.
Closes PR 14978
For detecting and debugging accidental dependencies on hash-table
order, it might be helpful to invert the order at the lowest level. To
do that, uncomment `#define REVERSE_HASH_TABLE_ORDER` in "hash.c".
The macro expander formerly put all lifted requires at the start of a
module, but that doesn't work with re-expansion if a module has
submodules and lifted requires that refer to submodules. Put lifted
submodules in the right place, instead: just before the form whose
expansion added the lifted require.
Racket wasn't reparsing correctly; the strategy worked ok
for links created by `mklink`, but not with other tools that
leave the "printed name" field blank.
A consequence of various fixes is that reparse points like
"My Documents" (in a typical configuration) correctly resolve
to actual paths like "Documents".
Finally, `directory-exists?` didn't handle root directories like
"C:/" correctly. The query would actually report properties of
the OS-level current working directory, and when junctions are
involved, the current directory can be a link instead of a directory.
Relevant to PR 14950 and PR 14912
Unlike `collapse-module-path`, it makes sense for
`collapse-module-path-index` to convert a relative module path index
to a plain module path. In other words, `collapse-module-path-index`
can convert a module path index to a module path.
Optimization can cause a `lambda` that was going to refer to a
top-level variable or syntax object to not refer to it after all.
Ideally, the prefix should be dropped from the closure, but
the change here is more conservative: it fixes the `lambda`s
annotation that's used by the GC to indicate that nothing will
be used from the prefix.
For GC purposes, if a "prefix" (a closure frame that caprues
top-level or module-level bindings) may refer to syntax objects
that are not used by any reachable closure, in which case the
syntax object can be dropped. This pruning of syntax objects
uses the infrastructure already in place to prune variables.
Syntax objects were not included in the original pruning
implementation, because they are unlikely to create
finalization cycles in the way that global-variable
references can. A syntax object can retain a namespace's
table of module imports, however, which can be substantial
and worth releasing of a closure is only held, say, for
a low-level finalization action.
Although names were cleared correctly, the trie used for
the mapping was not pruned correctly, so lots of empty
branches could accumulate (especially in 64-bit mode).
Even when `(variable-reference-constant? (#%variable-reference ....))`
cannot be optimized to a boolean, the expression should not retain a
reference to the enclosing namespace. That space guarantee is
important for the compilation of calls to keyword-accepting functions.
The handling of `for-template` imports by `namespace-attach-module`
didn't match the docs. The actual handling was to refrain from
attaching instances of a phase-0 module if the instance was reachable
only through a `for-template`. The rationale had to do with such
modules instances being created only through instantiation of
phase-1 modules, and phase-1 module instances aren't attached;
it doesn't work well that way, though, when different modules
are attached with intervening `namespace-require`s on the target
namespace.
The change includes a documentation correction. Previously and still,
only modules at the same phase as the attached module (as opposed to
the same phase or less) are instantiated in the target namespace.
Closes PR 14938
If a file or directory delete fails, try adjusting the file or directory
permissions to allow writes, then try deleting again. This process should
provide a more Unix-like experience and make programs behave more
consistently.
A new `current-force-delete-permissions` parameter provides access to
the raw native behavior.
Check for an empty path after dropping `"`s, instead of before.
Otherwise, a bad PATH setting interferes with functions like
`find-executable-path`, which in turn can prevent DrRacket from
starting up.
Closes PR 14930
If the slow path has to be taken because the number of
list elements is greater than the stack size, then the
old implementation would copy all the arguments --- which
still might be too much for the available stack space.
Avoid that copy.
Also, add pad word to the end of the stack to help detect
overflow.
For example, reduce (begin x (error 'e) y) ==> (begin x (error 'e)) and
(f (error 'e) y ) ==> (begin f (error 'e)).
Also, reduce (if (error 'e) x y) ==> (error 'e) and propagate the type information
and clocks when only one branch produce an error.
- Modify the features used by OpenBSD (not everything was
tested). Mostly copied from Linux, FreeBSD and NetBSD.
- Add support for Bitrig, a fork of OpenBSD. Eventually
they will differ more and more from OpenBSD.
- Typos and extra trailing spaces.
- Update config.guess and config.sub from GNU.
The implementation of caching stack-trace information in the
stack didn't work right in libunwind mode, with the result that
`(current-continuatiom-marks)` took O(N) time for a continuation
of size N, when it should be amortized constant time.
A value-printing truncation discovered after a stack-overflow handle
and return could go badly, because the truncation escape wasn't
reset correctly after overflow handling (in contrast to truncation
discovered during the overflow handling, which was handled correctly).
Closes PR 14870
Since `begin0` at the bytecode level always evaluates an initial
expression in non-tail position, we don't have to work so hard
to ensure that an extra expression sticks around.
Move begin0 inside begin, for example
(begin0 (begin X Y) Z) ==> (begin X (begin0 Y Z))
Try to replace more begin0 with begin when the first expression is movable
Drop the begin0 when it has only one non omitable expression that preserves
the continuation marks.
The source to the split packages is in repositories under the
`racket` organization on GitHub. The repositories are all named
according to the pkg name, except for multiple-package
repositories such as `racket/compiler` which is named based on the
old directory name without the `-pkgs` suffix. Thus
`pkgs/compiler-pkgs` -> https://github.com/racket/compiler
The Makefile has also been adjusted to pull packages from the
catalog when you type `make`. This currently relies on some tricks
that will break if you try to specify a particular set of `PKGS` on
the command line. We plan to improve this soon.
The packages in `pkgs/racket-pkgs` and `pkgs/base` are staying in
the repository, since they logically belong with the core code.
The `plt-services` package is still in the repository, but will
move out soon.
The optimizer converts (car (cons X Y)) to (begin0 X Y) and then reduces
it to (begin Y X) if X is movable.
Check that the movement is safe for space and for continuation captures.
When continuation C2 extends continuation C1, C2 shares the copy
of the internal stack with C1. It needs to skip the bit of
C1's stack that corresponds to arguments to `call/cc`, though.
That skipping assumed that `call/cc` takes 1 argument, but it can
take 2. The bug broke `racklog`, which captures continuations using
its own prompt. (It seems like there should be a simple test that
is independent of Racklog, but I couldn't construct it.)
Meanwhile, the continuation shouldn't retain the arguments to
`call/cc`, so clear them. (That was easy to test.) Sharing still
has to compensate for the locations of the arguments, though.
Fix the "self" argument propagation through an impersonator that has
no redirection function (but that probably has impersonator
properties).
Closes PR 14852
The new variants pass a "self" argument to the wrapper procedure in
the same way that `{impersonate,chaperone}-struct` provides a "self"
argument to redirection procedures.
A large message that hasn't been delivered can trigger a inter-place
GC. The intent is to force a GC to avoid messages piling up that can
never be delivered, but the GC didn't adjust to a state where messages
stay both undelivered and uncollected, and it would continuosly
trigger GCs. Trigger a GC only if the pending-message size has grown
relative to the previous GC.
Some expression movements are limited by the possibility of retaining
a value in a way that interacts with space safety, but primitives that
return immediately shouldn't get in the way of those movements.
When a variable X is bound to an expression that implies properties of
other bindings, and if X is used only once and can be replaced by
its value expression, then further optimization of that expression must
not assume the properties that are established by evaluating the
expression.
Also, don't move expressions past unsafe operations, since the expression
might implicitly guard against unsafety.
Closes PR 14819
I think that `-static-libgcc` didn't solve any problems with gcc
3.7.x, but with 3.8.x, divdi3() shows up, and that leads to
a "libgcc_s.dll" dependency unless `-static-libgcc` is used.
Fix various configuration problems, and make the build work with 3m
(probably for the first time).
The repairs include corrections for the manual link table, but also
switch Cygwin to relying on normal DLL exports, instead, to work
properly with the FFI.
The `--enable-shared` comfiguration option is no longer required for
Cygwin. When it is used, the `gracket` launcher does not work right,
because the Cygwin DLL is in the "bin" directory and "gracket.exe" is
in the "lib" directory. Along similar lines, stand-alone executables
won't work with `--enable-shared`.
The change to `ffi/winapi` makes it match the documentation.
These two functions allow the creation of relays that receive events
on logger B where there are interested receivers for logger A.
Based on comments from Tony Garnock-Jones.
Events to propagate to a parent are described in the same way
as events to receive for a log receiver. The default is still
to propagate all events to the parent, which corresponds to
a propagation specification of 'debug.
Making a propagation-filtering specification built-in, instead of
allowing arbitrary filter functions, keeps `log-level?` efficient and
avoid hooks that might be implemented by untrusted code.
The optional callback argument was added (by me) in f2d87085. This is
a backward-incompatible change, but allowing an arbitrary callback
on a logger now seems like an especially bad idea; forms like
`log-error` otherwise work in constrained contexts, while an arbitrary
callback function allows potentially untrusted code in those contexts.
Meanwhile, the addition doesn't satisfactorily solve the original
problem, since it intereferes with `log-level?` and similar filters.
There seems to be a problem in kqueue() on Mac OS X for watching for
FIFO write availability, where adding a read event for the same FIFO
(at a different file descritor) can disable the write event.
Merge to v6.1.1
If a write to a non-blocking descriptor fails, then try again
with fewer bytes, since nothing in the spec write() seems to
promise writing partial amounts. In particular, writing to
a FIFO no Mac OS X might fail even if there are a few bytes of
space; as it happens, the select() function seems to compensate
and claim that such a FIFO is full, but kqeueue() doesn't.
The check that the current meta-continuation matches the captured one
would always fail (I think), since the current meta-continuation is
pruned on capture. Keep a weak link to the original meta-continuation
to enable detection of capturing a continuation that matches or
extends one that was previously captured.
Enabling sharing exposed a problem with the code that saves
continuation marks for partial sharing, since that implementation
became out of sync with the main implementation (so merge the
implementations).
When calling a wrapper procedure for a field accessor or mutator,
provide the structure that was originally passed to the accessor or
mutator, instead of the value that was wrapped to create an
impersonator.
This is a backward-incompatible change, but I can't find any uses of
that initial argument to the wrapper procedure. Also, a wrapper can
capture the original value in its closure, while passing "self" allows
wrappers that are sensitive to overridden impersonator properties.
The OS doesn't necessarily react to a zero-sized buffer the way
that `udp-receive!` is supposed to work, so provide only a
non-zero-sized buffer to the OS.
In particular, a #f argument can make sense if the length is 0.
Technically, a byte string's byte array is supposed to be nul-terminated,
but many uses of byte strings get away without that terminator. I've
adjust the documentation to note that `bytes-copy` will work with a
non-terminated byte string.
Merge to v6.1.1
This bug could result in weird "cannot re-define a constant: lifted.0.2"
errors, or probably even worse collsisions of definitions, but I think
only in a namespace created from `module->namespace`.
So far, I haven't been able to create a reasonably small test,
because so many things have to line up in just the right way.
Merge to v6.1.1
If you set the locale to something like "us_EN.1252", then
"1252" was returned as the encoding, but "CP1252" is more likely
to be recognized by `bytes-open-converter`.
Also, document representation information on paths. In particular,
explain that Unix and Mac OS X paths are natively byte strings, while
Windows paths are natively UTF-16 code-unit sequences. The byte-string
representation of a Windows path is a UTF-8-like encoding of the UTF-16
code-unit sequence, which is why it makes no sense to convert it using
the current locale's encoding.
The `--enable-natipkg` configuration option adds "-natipkg" to the
platform library subpath. The suffix is intended to trigger the
installation of packages that supply native libraries for supported
platforms (where 64-bit Linux is the supported platform, for now, for
main-distribution packages), instead of relying on libraries installed
via the OS's package manager.
The intended client for "-natipkg" is the package-build service, where
installing packages via the OS package manager would require network
access and either trust or constrained installations. The build
machine is intentionally disconnected from the network and can only
access Racket packages, so repackaging native libraries as Racket
packages makes those libraries accessible.
A disadvantage of this approach to installing native libraries is that
it creates work for implementers of packages that access native
libraries. Those implementers will have to supply packages for 64-bit
Linux versions of native libraries to the degree needed to build and
(eventually) test the package. An advantage of the approach is that it
requires no changes to the package system; it will be cheap to replace
this approach if we find a better way to deal with native libraries
and/or OS packages in the package-build service.
Generalize `udp-send-to`, etc., to try each possibility of
a resolved address (instead of just the first one) like
`udp-connect!` does. This matters, for example, when using
"localhost" as an address, when the machine resolves "locahost"
to both "127.0.0.1" and "::1", and when the socket is created
for the second one that would be tried.
Also, detect and discard asynchronous ICMP errors.
The closure could be allocated as uninitialized memory with the
expectation that it would be filled right away, but boxing values
to put in the closure could expose the uninitialized memory to
the GC. Fix the problem by boxing before allocating closures.
On Windows, a "soft link" or "junction" is different from a
"symbolic link". The current Windows documentation is
incomplete in that it describes the behavior of GetFileAttributesEx
for a symbolic link, but not for a junction, and I guessed wrong.
For consistency, junctions need to be treated like symbolic links.
The `racket/draw` library is now independent of the screen resolution
on Windows. Font sizes in "points" are the only place where the
resolution mattered before, and now `racket/draw` assumes a
traditional 96dpi on Windows and Linux (and a traditional 72dpi
on Mac OS X).
Setting the scale for "text and other items" in Windows now adjusts
the backing scale of screen and canvas-compatible bitmaps, as well as
setting a scale on canvas drawing. Window and screen positions and
sizes are similarly scaled; for example, if the screen is 2048x1436
with text scaled by 200%, then `racket/gui` reports the display size
as 1024x768 (and the display backing scale as 2.0).
Backing scales of 1.25 and 1.5 are common for Windows. Rounding
associated with those scales could cause trouble for virtual -> actual
-> virtual conversions.
(as requested by Asumu)
A witness accessor or mutator is still required to create a structure
chaperone, but `#f` can be provided in place of a redirection, and
then impersonator properties can be attached to the chaperone.
At the same time, adjust `(chaperone-of? v1 v2)` so that `v1` as a
chaperone is not required to preserve non-redirecting chaperones of
`v2`.
The overall consequence is that a redirection procedure can cooperate
with a (suitably protected) impersonator property to override
redirection behavior without running afoul of the chaperone invariant
and without requiring O(N) space for O(N) overrides. For example, the
contract system can implement the re-application of a contract with
different blame information by overriding blame information as
represented by properties, instead of adding a new chaperone layer
every time that blame changes.
... and all the same for non-chaperone impersonators, of course.
While a foreigh call is normally guarded by a check on the amount
of available stack space, a callbacks triggered by the
scheduler will first put Racket in no-stack-overflow mode, and
then it's too late to check stack space before making further
foreign calls. With Cocoa, there's some chance that the process
will run out of space. Avoid the mismatch by checking the stack
availability at the start of a scheduler iteration.
Fixes a mistake in commit 768b93be82, which dropped a check that is
needed to trigger GCs during a sequence of large-block allocations.
Closes PR 14738
Refactor the code to move inside 'let' or 'begin'.
Also, in the test position of a 'if', recognize the 'not' inside a 'let' or 'begin'.
For example, transform (if (begin ... (not p)) x y) => (if (begin ... p) y x)
Previously, this conversion was made only when
the 'not' was the outermost expression.
And use the refactored code to move application inside 'let' or 'begin' in a single step
For example, transform ((let (...) ... (let (...) ... f) x) => (let (...) ... (let (...) ... (f x))
In the conversion, it's necessary to shift x to the new coordinates inside the 'let's.
In the new version x is shifted only once.
With `replace-evt` the time that the system needs to wake up
to check the event can drift later, but scheduling state was
carried in a way that works only if the wake-up time drifts
earlier.
Unfortunately, I don't know how to write a test for this bug.
The usual stategy of using `system-idle-evt` to detect busy
waiting doesn't work here, because the business happens despite
the scheduler's conclusion that the system is idle.
As reported by Jan Dvořák on the mailing list.
Fixes a problem with c4508ad0d9, which disabled module-code
caching too often. A symptom of the disabled cache was that
running "math/scribblings/math.scrbl" would use twice
as much memory.
For some types, (equal? x y) is transformed into (eq? x y) in the resolve phase.
This commit adds this transformation to the optimizer phase. This improves
constant folding and enable some optimizations that are prevented
because equal? can run arbitrary code.
Also, transform (eq? #f x) => (not x) and (eq? '() x) => (null? x) to use
the type information of x when it's known.
Currently the optimizer can convert ((let (...) ... proc) x) to
(let (...) ... (proc x)). This is useful especially if proc can be
inlined. Extend this to begin's forms.
Previously, the optimizer simplified the application of some unary functions inside let,
for example (car (let () ... (cons 1 2)) => (let () ... 1). This commit extends this to begin forms,
like (car (begin ... (cons 1 2)) => (begin ... 1).
Also, constant folding and some reductions were only availed in the direct case, for example
(procedure? car) => #t. With this commit these reductions are extended to the expressions
inside let and begin, for example (procedure? (let () (begin ... car))) => (let () (begin ... #t).
got broken in 2e284cc783
The racket version of libunwind is not compatible with QNX but old-style stacktraces are still working with the default gcc version
If "p" is available as a source package, which is typical, then `raco
pkg install --binary p` would strip away the build dependencies of "p",
so that "p" would not install properly.
This commit changes `raco pkg install` to look for an annotation on
the package and complain if the annotation is inconsistent with the
requested conversion: a binary package cannot be used as a source
package or vice versa. (A built package, as provided by a snapshot
site, can be used as any kind of package.)
Adjust dependency tracking and makefile rules to that when
`--enable-racket=...` is provided to `configure`, intermediate
CGC objects are not compiled.
The new approach uses dependency tracking that was already supported
by xform, previously used only for Windows.
Although newer versions of Visual Studio can open 2010 projects, the
meaning of the project turns out to be: use 2010 tools. So, I've added
a step in the build script to automatically upgrade the solutions and
projects based on the version of Visual Studio that is being run.
Meanwhile, since my previous tests for VS 2012 and VS 2013 were using
VS 2010 projects, I wasn't actually testing with the 2012 and 2013
compilers. Additional changes are needed to make those work, notably a
fresh implementation of setjmp() and longjmp() for Win64.
This was all very painful, but the projects are now in much better
shape, so maybe it won't be so bad from here.
The manifest was intended to enable XP-style controls, but at this
point it doesn't seem to do anything except interefere with some
variants of the build tools.
This new default for Unix and Mac OS X trades performance for
portability (hopefully), but for most users the switch affects only
for the build process, where `racketcgc` is used to build `racket`.
To continue using Boehm GC, configure with `--disable-sgc`.
For now, Boehm GC continues to be the default for Windows.
Allocation of executable memory is intended to make SELinux
happier by mmapping with PROT_EXEC instead of using mprotect()
to allow execution after the fact.
Performance improvements bring SGC within 30% of the Boehm GC on
`racketcgc -cl racket`, which makes SGC an even more plausible
substitute.
When a module is loaded with submodules intact, it should not be
cached and used for a later load that is intended to obtain the
module without submodules. Avoid mismatches by constraining the
cache to modules without submodules.
Error reports used the "source" field of a module, which
doesn't have submodule information, or the "name" field of
a module, which might not match an actual filename (".ss"
vs. ".rkt"). Create the right combination.
Adjust the compiler pass to insert checks for #<unsafe-undefined>.
The chanegs amount to throwing out the old attempt to follow the
implementation sketched in "Fixing Letrec", and instead use a
simpler abstract interpretation.
Bindings in `let` and `let*` need to be tracked much the same
way as for `letrec`, so that
(letrec ([b (let ([d (lambda () c)])
(d))]
[c 1])
b)
raises an exception.
A thread can be swapped out while it's in transition between a
mandling of the mark-stack position and recovering from C-stack
overflow. Fix up that case.
Previoulsy, `expand` mode explicitly disallowed cross-phase declaration
in commit 2e652fc2b3. I'm not sure why, and that commit has no test
case that fails when the restriction is removed, so my best guess is
that it was a debugging strategy that I forgot to undo.
The mz-gdbinit script (generated by mk-gdbinit.rkt) gives the type when using
pso, even when the default template did not include the type.
It defaults to printing out only the name of the type without additional
information.
Allows conversion of negative "seconds" to reach dates before
1970, and fixes year-varying DST tracking for versions of
Windows that know about those details.
As far as I can tell, we have to compute ourselves whether a
date is in daylight-saving time based on specifications of
when daylight and standard times start. That part seems tricky
and could use extra review.
Problem, example, and solution from Sam; see the dev mailing-list post
on 24-JUL-2014.
When a chaperoned accessor, mutator, or property accessor is used to
chaperone a struct, the chaproning procedure must not be able to
see things that the chaproned accessor, mutator, or property accessor
would not allow.
required that the identifier wrapped by `#%top` not have a local
binding. Change the documentation to match the implementation in that
way. (Since local binding in an identifier's lexical information
contributes to its identity as a top-level binding, that specification
of `#%top` would make sense everywhere, but I've left the top level
alone for backward compatibility.)
Also, change `local-expand` to never introduct `#%top`
wrappers. That's a little more consistent with what `#%top` has
evolved to mean, and it specifically works better with
`local-expand/capture-lifts`.
Closes PR 14635 and PR 14654
A `string-split` on a big string with lots of small matches sends the
regexp matcher a big string many times. Decoding 1024 bytes each time
is too much. Decoding 32 bytes is be a better trade-off between
chunking for large matches and being lazy for small matches.
For example, on a 60MB string with a space every 15 characters or so,
splitting on a space is about 3 times as fast with this adjustment.
I tried a few chunk sizes, and 32 worked the best in my experiments.
Naturally, as more bytes are read, the chunk size ramps up, so it's
a question of initial size; larger matches are relatively insensitive to
the initial size (so, again, it makes little sense to cater to large
matches with a large initial decoding size of 1024 bytes).
Cuts about 1/3 of the time for a string-hashing microbenchmark
provided by Pedro Ramos:
#lang racket
(define alphabet "abcdefghijklmnopqrstuvwxyz")
(define (random-word n)
(build-string n (lambda (x) (string-ref alphabet (random 26)))))
(define words (for/list ([k 1000000])
(random-word 3)))
(define d (make-hash))
(time (for ([w (in-list words)])
(if (hash-has-key? d w)
(hash-set! d w (add1 (hash-ref d w)))
(hash-set! d w 1))))
When `x` and `x`-with-a-mark are both defined, then the order of
definitions affected the binding that `(provide x)` would export
in a submodule that uses `#f` as its language. The problem was
in the implementation of the implicit `require`, which needs to
look up a variable's symbolic name in two different environments
to set up the right mapping.
In the case that the current time equals exactly the timeout of
a `sync/timeout`, the Racket process could get stuck (using no CPU)
instead of continuing as it should.
How did we not find this before? Why am I suddenly able to replicate
the problem (i.e., hitting exactly the target timeout in the secheduler
at the millisecond granularity)?
Merge to v6.1
As suggested by Jan Dvořák.
The event created by `replace-evt` is a kind of event-gated
version of `guard-evt`. In particular,
(guard-evt thunk)
could be expressed as
(replace-evt always-evt (lambda (_) (thunk)))
Use `replace-evt` as a shortcut for the case when you want to
synchronize on either A or C, but you need to wait for B to get C.
You could wait on A+B and then, if B is selected, wait on A+C;
wrapping B with `replace-evt` to generate C is a kind of shortcut that
is eaiser to write and avoids tear-down and re-setup of A.
The `replace-evt` constructor is more than a shortcut in the sense
that it builds the pattern A+B->A+C into `sync`, which enables
abstractions that need a B->C transition. So, `replace-evt` adds
expressiveness, but (perhap reassuringly) it does not add any new
rendezvous capability.
Naturally, the procedure given to `replace-evt` can produce
another `replace-evt`, and the event argument to
`replace-evt` could also be a `replace-evt`.
This bug (in xform, really) appears to be responsible for recent "JIT
buffer overflow" crashes. It could also cause other memory-corruption
crashes.
The bug could be triggered by any program that uses operators like
`+`, `<`, and `bitwise-ior` on more than 2 and less than 6 operands
(which is a lot of programs), but only if a certain allocation and
GC pattern happens at just the right time (which is why a crash was
relatively rare).
Merge to v6.1
The `vector->values` function set up multiple return values
badly in the case that the given vector is chaperoned.
The problem could lead to NULL as results for `vector->values`.
Merge to v6.1
Applying to a large number of arguments once causes the run-time
system to maintain a too-large buffer for managing tail calls in
the future. Decay the buffer size as it is reallocated.
Gemeralize Gustavo's change so that immediately-used right-hand sides
can be moved into any position that (like the binding context) enforces
single-valuedness --- for arbitrary right-hand expressions.
The protection against unsafe-function references was designed for
bytecode that referred to unsafe operations indirectly, and that
was broken when the compiler changed to refer to unsafe functions
directly in bytecode (to simplify JIT inlining bytecode optimization).
Actually, the relevant code (now removed) seems to be pointless,
since protected-binding checking should cover it already. Maybe
something else changed, or maybe the code was not properly checked
in the first place.
Now, `read` rejects a bytecode stream if it contains a direct
reference to an unsafe function and the code inspector is not the
original code inspector. It's still possible to synthesize bytecode
that contains an indirect reference, and then protected-binding
checking does its job.
Using select() to check whether a pipe is ready for writing seems to
fail on Mac OS X 10.8 and 10.9. See the PR for a small C program to
demonstrate. It's possible that the small program is broken and
there's no bug, but the program works on Linux and on Mac OS X 10.7
and 10.6.
Although poll() seems to work, switching completely to poll() is not a
good option on Mac OS X, since poll() does not support devices on that
platform.
The kqueue() facility seems to handle pipes and writing ok, so work
around the bug by enabling the use of kqueue() on pipes.
Closes PR 14596
Show the content of the buffer and the content of the temporary buffer
used to predict the size; then, I should be able to track down the
source of a mismatch.
This is the third try to fix the bug exposed by "fuzz.rkt". Previous
repairs addressed a symptom at the point of printing bad symbols,
instead of the cause at a failure to validate a symbol's encoding
when reading bytecode. This one fixes reading.
Commit 6a5a3037b4 was not quite right, because it used sightly the
wrong variant among a dozen decoding functions. The test suite caught
the problem, but I forgot to run it before pushing.
Also, repair the "Inside" documentation on the function that was
incorrectly used, and document the new variant.
Allow an effect-free `if` to not increment the effect-tracking
virtual clock (but increment the clock during branches, to avoid
moving computation into a branch).
Spend empty-`let`-elimination fuel more precisely, so that more
empty `let`s can be removed while still avoiding quadratic
compile times.
Convert
(let-values ([(<id> ...) (if <id-t>
(values <e1> ...)
(values <e2> ...))])
....)
to
(let ([<id> (if <id-t> <e1> <e2>)]
...)
....)
which duplicates the `(if <id-t> ....)` test, but that's likely to
be worthwhile to avoid multiple-values shuffling and enable more
constant and copy propagation.
A related improvement is to more eagerly discard `let` wrappers
with unused bindings during optimization, which could enable
further optimization, and allow moving conditionals relative
to other expressions to avoid intermediate binding.
Eagerly discarding `let` wrappers exposed a bug in the optimizer's
shifting of variable locations by exercising the relavant shifting
operation in shift_closure_compilation().
Closes PR 14588
While the commit comment in 3150b31eb7 still seems right, the changed
overlooked the fact that the arguments to a split `values` might get
reordered (due to the way binding positions are calculated). Fix the
optimizer to make sure that reordering is allowed.
The change touches a lot of code, because we want to use
"omittable" to implement "movable", and it's not ok to
reorder access of mutable variables (in case some other thread is
mutating them). We have to fix all the calls to "omittable"
to support the slight generalization.
The optimizer was willing to convert `(pair? (cons w (random)))` to
`(begin (random) #t)`, but not `(car (cons w (random)))` to
`(begin (random) w)` because the `(car (cons ....))` transformation
required simple ignored arguments. Put the treatment of ignored,
non-omittable arguments of a dropped operation in one place. Also,
recognize expressions within `begin` whose results will be ignored.
Windows parses relative-path links with yet another set of rules ---
slightly different from the many other existing rules for parsing
paths. Unfortunately, a few OS calls don't provide an option for
having the OS follow links, so we have to re-implement (our best guess
at) the OS's parsing of links.
While something like `(if 7 e1 e2)` was already reduced to `e1`,
this improvement makes `(if (let ([x (random)]) 7) e1 e2)`
reduce to `(if (let ([x (random)]) #t) e1 e2)`.
The previous build attempted to cooperate with Valgrind in a way that
truncates a 64-bit address to a 32-bit address. Disable Valgrind
cooperation. (Other builds seem ok for now, but future rebuilds will
disable Valgrind cooperation for them, too.)
Visual Studio 2008 is still supported by "9.sln" projects and
".vcproj" files, while ".sln" and ".vcxproj" work for 2010, 2012,
and 2013. The "build.bat" script detects which version of
Visual Studio is active and uses the appropriate ".sln" files.
The bad news is that we have two copies of each project until we
decide to drop support for Visual Studio 2008. (We had two copies
before, but one copy was unmaintained.) The good news is that
we have only 2 copies of each project, because recent versions of
Visual Studio are willing to use 2010 projects as-is.
Change project and related files to text instead of always CRLF,
because that seems to be ok for modern Windows tools.
Calling the default module name resolver with three arguments logs an
error message. The intent is that 3-argument support will be removed,
eventually.
