Make the old-generation marking process incremental
on request, where `(collect-garbage 'incremental)`
makes a request.
Only the marking phase of an old-generation collection
is incremental, so far. In exchange for slower
minor collections and a larger heap, you get a major
collection pause time that is roughly halved. So, this
is a step forward, but not good enough for most purposes
that need incremental collection.
An incremental-mode request sticks until the next
major GC. The idea is that any program that could
benefit from incremental collection will have
some sort of periodic task where it can naturally
request incremental mode. (In particular, that
request belongs in the program, not in some external
flag to the runtime system.) Otherwise, the
system should revert to non-incremental mode, given
that incremental mode is slower overall and can
use much more memory --- usually within a factor of
two, but the factor can be much worse due to
fragmentation.
For a minor GC and pages that contain backpointers,
leave mark bits as they are; instead make a pass to
shift mark bits for new objects to "dead" bits, and
use dead bits for fixup.
This change is intended as a small step toward incremental
collection.
The rule for using generation 1/2is based on the current
memory use versus the maximum size of generation 0. Recent
changes to the GC have caused that size to vary during
a collection, which means that the choice to use generation
1/2 or not can change within a collection.
Partial use of generation 1/2 doesn't inherently cause problems, but
it can cause a generation-1 object to point to a generation-1/2 object
even though the former was allocated after the latter. That's a
problem on if getting generations out of order relative to allocation
order can create problems. As it happens, reset_finalizer_tree()
checks the generation of the finalization record and not the finalized
pointer, because the record is always allocated after the pointer.
Merge to v6.3
Certain datatypes in the runtime system are not supposed
to be hashed, where bits normally reserved for hash codes
are used for other purposes. A bad bytecode file can cause
some of those to be hashed, anyway. Normally, the damage is
isolated to that content of the damaged bytecode, but
certain variable-reference bytecode forms are both shared
and non-hashable. Set a bit that ensures hashing will not
change flags in the shared object.
This problem was exposed by fuzz testing.
History of the parsing of "file:" URLs for Windows:
* In response to PR 8060 (April 2006): special handling added to
support ill-formed URLs that were (are?) commonly used for
filesystem paths.
* Follow-up to PR 8060 (April 2008): added `path->url` and
`url->path`.
* In response to #1086 (October 2015, the commit): changed
Windows-specific handling to be more constrained and added support
for the proper encoding of UNC paths --- where "proper" means
"according to a blog post from late 2006", which appears to be as
close as we get to documentation of the URL encoding for Windows
paths.
When a compiler is run in standards mode, predefined macros that
do not start with "_" are dropped, so use the "_" versions
consistently. Whether or not Racket itself would compile in
standards mode, the Racket headers should be able to work that
way --- at least on Unix platforms.
In Mac OS X 10.11, something about the use of exceptions triggers
a libunwind stack traversal, and that traversal runs into trouble
with Racket's stack mangling for threads. Inserting generated code
in the stack frame sequence causes libunwind to give up and avoids
a crash (e.g., with `-j -l drracket` on startup).
After some expansions, a expression with the syntax property 'inferred-name of
'x is converted to one with ('x . 'x), so it's not useful to get the name of a
procedure. So we simplify the syntax property 'inferred-name to handle
these cases.
When a place message is deserialized by simply adopting the page
containing the message, the adoption can trigger a garbage
collection, but there's still a pointer to a chain of objects
"in flight" in the thread, and a GC can discard the pairs that
form the chain.
Removing all original module context doesn't work, because it
doesn't distinguish between fragments of syntax that had the
"inside-edge" scope without the "outside-edge" scope.
Record the presence of the outside-edge scope by using the
root scope, and convert the root scope to the current namespace's
outside-edge scope on evaluation.
The bug could cause
#lang racket/base
(define x 'outer)
(define-syntax-rule (def-and-use-m given-x)
(begin
(define-syntax-rule (m)
(let ()
(define given-x 'inner)
x))
(m)))
(def-and-use-m x)
to produce 'inner when it should produce 'outer.
Thanks to Brian Mastenbrook for pointing the problem and
providing examples.
Interrupting bytecode unmarshal for syntax objects could leave
half-constructed values in a table that is intended to resolve graph
structure. Clear out work towards a graph construction when
interrupted.
The most common symptom of half-constructed syntax objects was a crash
after a Ctl-C during startup.
Rename fields in a page record and split some of them with `union` to
better document the intent of each field.
This change is intended to have no effect on the GC's behavior. One
tricky case is the line dropped around line 3542 of "newgc.c". That
line reset `scan_boundary` (formerly `previous_size`), which on the
surface is inconsistent with leving objects before the boundary
without `marked` bits set. However, that line is reachable only
when geneation-1 objects are being marked (objects newly moved
there would not be unmarked), in which case `san_boundary` should
already be reset.
Using `--enable-racket=auto` causes a Racket for the current platform
to be built in a "local" subdirectory of the build directory as
support for cross-compilation.
The original idea was to count phantom bytes as "administrative
overhead", but issues discussed in #962 identified problems
with that idea. Finish shifting the accounting to treat
phantom bytes as payload allocation.
The misplacement of `SCHEME_PRIM_SOMETIMES_INLINED` caused the
optimizer to produce different results when the JIT is statically
disabled, for example.
This bug is an old one, in a sense, because travesing fields
in a closure could have moved the prefix with earlier versions
of the collector. It shows up now because we're changing fields
one indirection closer.
Compact fewer blocks by moving them only when other
blocks have room.
Also, fix block protection tracking in the case of a page
count that isn't divisible by 8.
In the common case of a minor GC without a generation 1/2
or a major GC without compaction, a single pass suffices
to both mark and update references.
This change reduces overall GC time by 10%-25% on typical
programs.
The GC supported allocation for an array of objects where
the first one provides a tag, but at this point it was
used only in some corners. Change those corner and simplify
the GC by removing support for arrays of tagged objects.
The main corner to clean up is in the handling of a macro-expansion
observer and inferred names. Move those into the compile-time
environment. It's possible that name inference has been
broken by the changes, but in addition to passing the tests,
the generated bytecode for the base collections is exactly the
same as before the change.
Although a block cache is set up to group most page-protection changes
into a single OS call, allocating new old-generation pages was not
covered. Adjust the block cache to group those.
This change has a small effect on performance, but it seems
better to have a few system calls in place of thousands.
First bug:
When the optimize converts
(let-values ([(X ...) (values M ...)])
....)
to
(let ([X M] ...)
....)
it incorrectly attached a virtual timestamp to each "[X M]" binding
that corresponds to the timestamp after the whole `(values M ...)`.
The solution is to approximate tracking the timestamp for invidual
expressions.
Second bug:
The compiler could reorder a continuation-capturing expression past
an allocation.
The solution is to track allocations with a new virtual clock.
Make `eval-syntax`, `compile-syntax`, and `expand-syntax` more
consistent (with intent and each other) by not installing a fallback
automatically. In particular, a fallback is not installed for a
`module` form, so that different ways of expanding a `module` form
produce consistent results (e.g., for ambiguous bindings).
p ..k matches exactly k repetitions of p, but its documented
behavior is to match "k or more" repetitions. This fix implements
the documented behavior.
Fixes bug number 15122.
Also, strengthen the checking that `#:permissive?` (off by default)
performs for `untar` and `untgz` to disallow a link whose target is an
absolute path or has an up-directory element.
Retry communication up to five times when `exn:fail:network`
is raised.
Not all `exn:fail:network` exceptions are transient. For example,
attempting to connect to a bad server name will produce an error more
slowly than before, since the bad connection will be tried five times.
Still, retrying on `exn:fail:network` seems like a good heuristic.
This fixes the DrRacket check syntax arrows in uses of `attribute` like this:
```racket
#lang racket/base
(require syntax/parse)
(syntax-parse #'a
[a
(attribute a)])
```
The symbol is used as the "who" field in the error message.
Also fix lazy-require of runtime-report.rkt in residual.rkt; don't
load until syntax-parse actually needs to produce an error report.
(Previously was loaded to create handler whenever syntax-parse code ran.)
When `raco pkg {install,update}` has a list of packages to check,
first run through the list in a "prefetch" mode to create futures to
fetch the information. Issuing requests to servers in parallel
can greatly speed up `raco pkg update --all`.
Package content is still downloaded sequentially.
Using the GitHub API for GitHub sources can run afoul of API
limits. Since we now support the Git protocol generall, use
that for GitHub sources, too.
Set the `PLT_USE_GITHUB_API` environment variable to use the
GitHub API, instead.
Putting <PlatformToolset> in the new places makes the projects
work when more than one version of Visual Studio is installed.
Maybe the old place was always the wrong place, or maybe
VS 2010 wanted it in the old place. Either way, sprinkling
the version in more places seems unlikely to hurt.
The SQLITE_READONLY_ROLLBACK error is supposed to mean that a crash
occurred and a hot journal exists that needs to be replayed (but
can't, because the current connection is read-only).
In practice, it seems that the error can happen even if there has been
no crash. In that case, retrying in the same was as other transient
errors allows the process to continue.
A possible reason for the spurious error: In the implementation of
SQLite, comments in hasHotJournal() mention the possibility of false
positives (ticket #3883) and how the false positive will be handled in
the playback mechanism after obtaining an exclusive lock. The check
for a read-only connection after hasHotJournal() is called, however,
happens before that lock is acquired. So, it seems like the race
condition could trigger a false SQLITE_READONLY_ROLLBACK error.
User-scope package installation matching the version of
Racket being built could affect the collections visible
during `raco setup` for `make base`. In particular, the
presence of `setup/scribble` could cause all built docs
to be discarded.
Also, add the `--no-user-path` flag to `racket` (which
has long been documented as an alias for `-U`).
The table as a tree is traversed to prune empty branches,
but the travseral is needed only toward branches that
have changed. Skipping the traversal can save several
milliseconds on each collection.
Name handling formerly interned symbols along the
way to allocating a plain string, which takes effort
and causes changes to the symbol table, which forces
a minor GC to traverse the whole symbol table. Skip
unnecessary symbol-interning steps.
Refine the changes in 16c198805b so that `(define id ... id ... )` at
the top level compiles more consistently when `id` is an identifier
whose lexical context does not include `#%top`.
When `compile` is used on a top-level definition, do not
create a binding in the current namespace, but arrange for
a suitable binding to be in place for the target namespace.
Closes#1036
This repair adjusts the bug fix of commit 769ad3e98. That older commit
ensured that `sync/enable-break` doesn't both break and accept a
channel message or semaphore wait. But it effectively disables those
actions if the break is continued.
Instead of (partially!) ending the `sync` get out of semaphore
and channel queues so that no event can be selected during
the break, and then get back in line if the break is continued.
When a path is made relative for marshaling to bytecode, record
a list of byte strings in stead of a platform-specific relative
path.
For syntax-object source locations, convert any non-relative path to a
string that shows just the last couple of path elements preceded by
".../". This conversion avoids embedding absolute paths in bytecode,
but at the cost of some information. A more complete and consistent
solution would invove using a module-path index instead of a path, but
that would be a big change at several layers.
Make room in the bytecode format for source locations and 'paren-shape
property values for syntax objects. Saving source locations increases
bytecode size by about 10% on average.
Also, convert the internal representation of syntax properties to
use immutable hash tables, instead of lists.
The `prop:expansion-contexts` property can control the expansion
of a rename transformer in much the same that conditionals on
`(syntax-local-context)` can control the expansion of other
transformers.
These avoid one layer of currying and are more efficient, getting
about a 1.3x speed up on this program:
#lang racket/base
(module server racket/base
(require racket/contract/base)
(provide
(contract-out
[f (-> integer? boolean? char? void?)]))
(define (f i b c) (void)))
(require (submod "." server))
(time
(for ([x (in-range 10000000)])
(f 1 #t #\x)))
All places uses the same accounting bit for objects
that are in the shared space. Each place also flips
the bit value it wants on each accounting, so if two
places are accounting at the same time with opposite
bit values and can reach the same objects, they can
interefere. It's even possible for them to race
through cycles and cause each other to loop forever.
Add a lock to ensure that there's only one bit value
in play for the shared space at any given time. A
place must stall if other places are busy with memory
accounting and an opposite bit value.
While a place message is received by a thread but not yet
deserialized, if the message contains references to objects in the
shared space, and if a "master" GC happens (which crosses all places),
make sure that the references in the still-serialized message are
traversed.
This is the most common contract created for -> (at 318 occurrences
out of the 6515 arrow contracts created when DrRacket starts up)
but this specialization doesn't seem to actually improve the performance
much. Leave it in for now, in case the story changes at some point
in the future
Adjust installation tools to support cross-installation (i.e.,
installation for a platform other than the current one) as triggered
by "system.rktd" in "lib" having different information than the
running Racket executable.
Also, change floating-point handling to be like the MSVC build by
default, where the process is left in double-precision mode and
the mode is changed for exfl operations.
Includes repairs for integer-size mismatches in uses of Windows
threads.
The error message for the guard used an incorrect contract.
Also removed an unused line that allows a box value in the
property. I don't think it was possible to trigger this line
anyway because of the dynamic check.
Support PNG-encoded icons in ".ico" files and executables.
For executables, instead of supporting only new icons that match the
sizes and encodings of existing icons in an executable, support
arbitrary replacement icons in an executable.
The improved funcitonality relies on a new library (currently
private) for general updates to a Windows executable's
resources.
In a case like
(let-values ([(X ...) (with-continuation-mark M_k M_v
(values M ...))])
....)
where the bytecode compiler cannot convert to a sequence of `let`
bindings, make the JIT implement `values` as delivering argument
results directly to the corresponding variable locations.
When `call/cc` and `call/ec` were moved out of `#%kernel`, the
`let/cc` and `let/ec` macros changed to refer to
`call-with-current-continuation` and `call-with-escape-continuation`.
Move `call/cc` and `call/ec` again and restore the macros, so that
matching on the expansion of the macros (e.g,. in the web server's
language that looks for `call/cc`) work as before.
A `module-suffixes` entry in a collection's "info.rkt" adds a
file suffix that is meant to be recognized globally (i.e., in
all collections) by all Racket tools.
The new fields are reported by `compiler/module-suffix` library, which
is (so far) used by `raco setup`.
Note that if package A includes files with a suffix that is registered
by package B, then A should have a dependency on B, but `raco setup`
cannot currently detect that such a dependency is needed. That
dependency is likely to happen, anyway, since package A is likely
using libraries form package B.
Comments in the implementation suggest that a timestamp
specialization was intended for directories, which makes
sense given that directory-modification dates are
not preserved when unpacking an archive. The change
also affected all bytecode timestamps, however, which
can create inconsistencies across package creations.
The `from` string argument is converted to a regexp and cached. When `from` is
a mutable string this can cause wrong results in the following calls
to string-replace. So the string is first converted to an immutable string to
be used as the key for the cache.
When `place` expands, the body of the `place` form is placed into a
`(module* place-body-<n> #f ....)` submodule.
The `place` form previously placed its body in a lifted function,
where the function's exported name was based on
`(current-inexact-milliseconds)`. The generated submodules have
deterministic names, so that compilation is deterministic, and
submodule names don't collide (unlike exported function names) when
multiple `place`-using module are imported into some other module.
Also, using a submodule avoids the problem that the clock doesn't
change fast enough on Windows.
Those aliases were moved out of `#%kernel` as part of the
determinstic-bytecode changes, but putting them in
`racket/private/pre-base` meant that they weren't included in
`mzscheme` or Pretty Big. The new location is with `let/cc`, which
makes more sense, and makes them picked up by `mzscheme` and Pretty
Big.
`net/url` provides functions for both converting strings
and paths to and from URLs.
`net/url` also includes functions for creating (pure and import)
network ports. This functionality `require` the HTTP client stack
which is unnecessary when URLs simple need parsing for their
"bits".
New library: `net/url-strings` handles `url->string` and `string->url`
(and also the related `path->url` and `url->path` functions). This is
required by net/url for compatability.
`net/url-exception.rkt` is factored out for use by both libraries.
- See also racket/net changes for T&D
url-string.rkt changes requested by mflatt
url-strings.rkt is now called url-string.rkt
identifiers from url-string.rkt are reprovided by url.rkt
using (all-from-out "url-string.rkt") instead of explicit
exports
