37a985a681
The expander as a linklet will be instantiated once, so there's no need to capture references in closures among functions within the expander. Add a "static" linklet compilation mode to inline the variable addresses that would otherwise be referenced via a closure. Although the change is intended to speed up the expander by avoiding some indrections, it also reduces the bytecode size of the expander. Bitmaps that track which linklet variables are used in closures turn out to have been about 25% of the expander's bytecode size, since the linklet has so many definitions.
469 lines
21 KiB
Racket
469 lines
21 KiB
Racket
#lang scribble/doc
|
|
@(require "mz.rkt"
|
|
(for-label racket/linklet
|
|
racket/unsafe/ops))
|
|
|
|
@title[#:tag "linklets"]{Linklets and the Core Compiler}
|
|
|
|
@defmodule[racket/linklet]
|
|
|
|
A @deftech{linklet} is a primitive element of compilation, bytecode
|
|
marshaling, and evaluation. Racket's implementations of modules,
|
|
macros, and top-level evaluation are all built on linklets. Racket
|
|
programmers generally do not encounter linklets directly, but the
|
|
@racketmodname[racket/linklet] library provides access to linklet
|
|
facilities.
|
|
|
|
A single Racket module (or collection of top-level forms) is typically
|
|
implemented by multiple linklets. For example, each phase of
|
|
evaluation that exists in a module is implemented in a separate
|
|
linklet. A linklet is also used for metadata such as the @tech{module
|
|
path index}es for a module's @racket[require]s. These linklets, plus
|
|
some other metadata, are combined to form a @deftech{linklet bundle}.
|
|
Information in a @tech{linklet bundle} is keyed by either a symbol or
|
|
a @tech{fixnum}. A @tech{linklet directory} containing
|
|
@tech{linklet}s can be marshaled to and from a byte stream by
|
|
@racket[write] and (with @racket[read-accept-compiled] is enabled)
|
|
@racket[read].
|
|
|
|
When a Racket module has submodules, the @tech{linklet bundles} for
|
|
the module and the submodules are grouped together in a
|
|
@deftech{linklet directory}. A @tech{linklet directory} can have
|
|
nested linklet directories. Information in a linklet directory is
|
|
keyed by @racket[#f] or a symbol, where @racket[#f] must be mapped to
|
|
a @tech{linklet bundle} (if anything) and each symbol must be mapped
|
|
to a @tech{linklet directory}. A @tech{linklet directory} can be
|
|
equivalently viewed as a mapping from a lists of symbols to a
|
|
@tech{linklet bundle}. Like @tech{linklet bundles}, a @tech{linklet
|
|
directory} can be marshaled to and from a byte stream by
|
|
@racket[write] and @racket[read]; the marshaled form allows individual
|
|
@tech{linklet bundles} to be loaded independently.
|
|
|
|
A linklet consists of a set of variable definitions and expressions,
|
|
an exported subset of the defined variable names, a set of variables to export
|
|
from the linklet despite having no corresponding definition, and a set
|
|
of imports that provide other variables for the linklet to use. To run
|
|
a linklet, it is instantiated as as @deftech{linklet instance} (or
|
|
just @defterm{instance}, for short). When a linklet is instantiated,
|
|
it receives other @tech{linklet instances} for its imports, and it
|
|
extracts a specified set of variables that are exported from each of
|
|
the given instances. The newly created @tech{linklet instance}
|
|
provides its exported variables for use by other linklets or for
|
|
direct access via @racket[instance-variable-value]. A @tech{linklet
|
|
instance} can be synthesized directly with @racket[make-instance].
|
|
|
|
A linklet is created by compiling an enriched S-expression
|
|
representation of its source. Since linklets exist below the layer of
|
|
macros and syntax objects, linklet compilation does not use
|
|
@tech{syntax objects}. Instead, linklet compilation uses
|
|
@deftech{correlated objects}, which are like @tech{syntax objects}
|
|
without lexical-context information and without the constraint that
|
|
content is coerced to correlated objects. Using an S-expression or
|
|
@tech{correlated object}, the grammar of a linklet as recognized by
|
|
@racket[compile-linklet] is
|
|
|
|
@specform[(linklet [[imported-id/renamed ...] ...]
|
|
[exported-id/renamed ...]
|
|
defn-or-expr ...)
|
|
#:grammar
|
|
([imported-id/renamed imported-id
|
|
(external-imported-id internal-imported-id)]
|
|
[exported-id/renamed exported-id
|
|
(internal-exported-id external-exported-id)])]
|
|
|
|
Each import set @racket[[_imported-id/renamed ...]] refers to a single
|
|
imported instance, and each @racket[_import-id/renamed] corresponds to
|
|
a variable from that instance. If separate
|
|
@racket[_external-imported-id] and @racket[_internal-imported-id] are
|
|
specified, then @racket[_external-imported-id] is the name of the
|
|
variable as exported by the instance, and
|
|
@racket[_internal-imported-id] is the name used to refer to the
|
|
variable in the @racket[_defn-or-expr]s. For exports, separate
|
|
@racket[_internal-exported-id] and @racket[_external-exported-id]
|
|
names corresponds to the variable name as exported as referenced
|
|
in the @racket[_defn-or-expr]s, respectively.
|
|
|
|
The grammar of an @racket[_defn-or-expr] is similar to the expander's
|
|
grammar of fully expanded expressions (see @secref["fully-expanded"])
|
|
with some exceptions: @racket[quote-syntax] and @racket[#%top] are not allowed;
|
|
@racket[#%plain-lambda] is spelled @racket[lambda];
|
|
@racket[#%plain-app] is omitted (i.e., application is implicit);
|
|
@racket[lambda], @racket[case-lambda], @racket[let-values], and
|
|
@racket[letrec-values] can have only a single body expression; and
|
|
numbers, booleans, strings, and byte strings are self-quoting.
|
|
Primitives are accessed directly by name, and shadowing is not allowed
|
|
within a @racketidfont{linklet} form for primitive names, imported
|
|
variables, defined variables, or local variables.
|
|
|
|
When a @racket[_exported-id/renamed] has no corresponding definition
|
|
among the @racket[_defn-or-expr]s, then the variable is effectively
|
|
defined as uninitialized; referencing the variable will trigger
|
|
@racket[exn:fail:contract:variable], the same as referencing a
|
|
variable before it is defined. When a target instance is provided to
|
|
@racket[instantiate-linklet], any existing variable with the same name
|
|
will be left as-is, instead of set to undefined. This treatment of
|
|
uninitialized variables provides core support for top-level evaluation
|
|
where variables may be referenced and then defined in a separate
|
|
element of compilation.
|
|
|
|
@history[#:added "6.6.1"]
|
|
|
|
@; --------------------------------------------------
|
|
|
|
@defproc[(linklet? [v any/c]) boolean?]{
|
|
|
|
Returns @racket[#t] if @racket[v] is a @tech{linklet}, @racket[#f]
|
|
otherwise.}
|
|
|
|
|
|
@defproc*[([(compile-linklet [form (or/c correlated? any/c)]
|
|
[name any/c #f]
|
|
[import-keys #f #f]
|
|
[get-import #f #f]
|
|
[options (listof (or/c 'serializable 'unsafe 'static)) '(serializable)])
|
|
linklet?]
|
|
[(compile-linklet [form (or/c correlated? any/c)]
|
|
[name any/c]
|
|
[import-keys vector?]
|
|
[get-import (or/c #f (any/c . -> . (values (or/c linklet? instance? #f)
|
|
(or/c vector? #f))))
|
|
#f]
|
|
[options (listof (or/c 'serializable 'unsafe 'static)) '(serializable)])
|
|
(values linklet? vector?)])]{
|
|
|
|
Takes an S-expression or @tech{correlated object} for a
|
|
@schemeidfont{linklet} form and produces a @tech{linklet}.
|
|
As long as @racket['serializable] included in @racket[options], the
|
|
resulting linklet can be marshaled to and from a byte stream when it is
|
|
part of a @tech{linklet bundle}.
|
|
|
|
The optional @racket[name] is associated to the linklet for debugging
|
|
purposes and as the default name of the linklet's instance.
|
|
|
|
The optional @racket[import-keys] and @racket[get-import] arguments
|
|
support cross-linklet optimization. If @racket[import-keys] is a
|
|
vector, it must have as many elements as sets of imports in
|
|
@racket[form]. If the compiler becomes interested in optimizing a
|
|
reference to an imported variable, it passes back to
|
|
@racket[get-import] (if non-@racket[#f]) the element of @racket[import-keys] that
|
|
corresponds to the variable's import set. The @racket[get-import]
|
|
function can then return a linklet or instance that represents an instance to be
|
|
provided to the compiled linklet when it is eventually instantiated;
|
|
ensuring consistency between reported linklet or instance and the eventual
|
|
instance is up to the caller of @racket[compile-linklet]. If
|
|
@racket[get-import] returns @racket[#f] as its first value, the
|
|
compiler will be prevented from make any assumptions about the
|
|
imported instance. The second result from @racket[get-import] is an
|
|
optional vector of keys to provide transitive information on a
|
|
returned linklet's imports (and is not allowed for a returned instance);
|
|
the returned vector must have the same
|
|
number of elements as the linklet has imports. When vector elements
|
|
are @racket[eq?] and non-@racket[#f], the compiler can assume that
|
|
they correspond to the same run-time instance. A @racket[#f]
|
|
value for @racket[get-import] is equivalent to a function that
|
|
always returns two @racket[#f] results.
|
|
|
|
When @racket[import-keys] is not @racket[#f], then the compiler is
|
|
allowed to grow or shrink the set of imported instances for the
|
|
linklet. The result vector specifies the keys of the imports for the
|
|
returned linklet. Any key that is @racket[#f] or a @tech{linklet instance}
|
|
must be preserved intact, however.
|
|
|
|
If @racket['unsafe] is included in @racket[options], then the linklet
|
|
is compiled in @deftech{unsafe mode}: uses of safe operations within
|
|
the linklet can be converted to unsafe operations on the assumption
|
|
that the relevant contracts are satisfied. For example, @racket[car]
|
|
is converted to @racket[unsafe-car]. Some substituted unsafe
|
|
operations may not have directly accessible names, such as the unsafe
|
|
variant of @racket[in-list] that can be substituted in @tech{unsafe
|
|
mode}. An unsafe operation is substituted only if its (unchecked)
|
|
contract is subsumed by the safe operation's contract. The fact that
|
|
the linklet is compiled in @tech{unsafe mode} can be exposed through
|
|
@racket[variable-reference-from-unsafe?] using a variable reference
|
|
produced by a @racket[#%variable-reference] form within the module
|
|
body.
|
|
|
|
If @racket['static] is included in @racket[options] then the linklet
|
|
must be instantiated only once; in the linklet is serialized, then any
|
|
individual instance read from the serialized form must be instantiated
|
|
at most once. Compilation with @racket['static] is intended to improve
|
|
the performance of references within the linklet to defined and
|
|
imported variables.
|
|
|
|
The symbols in @racket[options] must be distinct, otherwise
|
|
@exnraise[exn:fail:contract].}
|
|
|
|
|
|
@defproc*[([(recompile-linklet [linklet linklet?]
|
|
[name any/c #f]
|
|
[import-keys #f #f]
|
|
[get-import (any/c . -> . (values (or/c linklet? #f)
|
|
(or/c vector? #f)))
|
|
(lambda (import-key) (values #f #f))])
|
|
linklet?]
|
|
[(recompile-linklet [linklet linklet?]
|
|
[name any/c]
|
|
[import-keys vector?]
|
|
[get-import (any/c . -> . (values (or/c linklet? #f)
|
|
(or/c vector? #f)))
|
|
(lambda (import-key) (values #f #f))])
|
|
(values linklet? vector?)])]{
|
|
|
|
Like @racket[compile-linklet], but takes an already-compiled linklet
|
|
and potentially optimizes it further.}
|
|
|
|
|
|
@defproc[(eval-linklet [linklet linklet?]) linklet?]{
|
|
|
|
Returns a variant of a @racket[linklet] that is prepared for JIT
|
|
compilation such that every later use of the result linklet with
|
|
@racket[instantiate-linklet] shares the JIT-generated code. However,
|
|
the result of @racket[eval-linklet] cannot be marshaled to a byte
|
|
stream as part of a @tech{linklet bundle}, and it cannot be used with
|
|
@racket[recompile-linklet].}
|
|
|
|
|
|
|
|
@defproc*[([(instantiate-linklet [linklet linklet?]
|
|
[import-instances (listof instance?)]
|
|
[target-instance? #f #f]
|
|
[use-prompt? any/c #t])
|
|
instance?]
|
|
[(instantiate-linklet [linklet linklet?]
|
|
[import-instances (listof instance?)]
|
|
[target-instance instance?]
|
|
[use-prompt? any/c #t])
|
|
any])]{
|
|
|
|
Instantiates @racket[linklet] by running its definitions and
|
|
expressions, using the given @racket[import-instances] for its
|
|
imports. The number of instances in @racket[import-instances] must
|
|
match the number of import sets in @racket[linklet].
|
|
|
|
If @racket[target-instance] is @racket[#f] or not provided, the result
|
|
is a fresh instance for the linklet. If @racket[target-instance] is an
|
|
instance, then the instance is used and modified for the linklet
|
|
definitions and expressions, and the result is the value of the last
|
|
expression in the linklet.
|
|
|
|
The linklet's exported variables are accessible in the result instance
|
|
or in @racket[target-instance] using the linklet's external name for
|
|
each export. If @racket[target-instance] is provided as
|
|
non-@racket[#f], its existing variables remain intact if they are not
|
|
modified by a linklet definition.
|
|
|
|
If @racket[use-prompt?] is true, then the evaluation each definition
|
|
and expression in the linklet is wrapped in a @tech{prompt} in the
|
|
same ways as an expression in a module body.}
|
|
|
|
|
|
@defproc[(linklet-import-variables [linklet linklet?])
|
|
(listof (listof symbol?))]{
|
|
|
|
Returns a description of a linklet's imports. Each element of the
|
|
result list corresponds to an import set as satisfied by a single
|
|
instance on instantiation, and each member of the set is a variable
|
|
name that is used from the corresponding imported instance.}
|
|
|
|
@defproc[(linklet-export-variables [linklet linklet?])
|
|
(listof symbol?)]{
|
|
|
|
Returns a description of a linklet's exports. Each element of the list
|
|
corresponds to a variable that is made available by the linklet in its
|
|
instance.}
|
|
|
|
|
|
@defproc[(linklet-directory? [v any/c]) boolean?]{
|
|
|
|
Returns @racket[#t] if @racket[v] is a @tech{linklet directory},
|
|
@racket[#f] otherwise.}
|
|
|
|
|
|
@defproc[(hash->linklet-directory [content (and/c hash? hash-eq? immutable? (not/c impersonator?))])
|
|
linklet-directory?]{
|
|
|
|
Constructs a @tech{linklet directory} given mappings in the form of a
|
|
@tech{hash table}. Each key of @racket[content] must be either a
|
|
symbol or @racket[#f], each symbol must be mapped to a @tech{linklet
|
|
directory}, and @racket[#f] must be mapped to a @tech{linklet bundle}
|
|
or not mapped.}
|
|
|
|
|
|
@defproc[(linklet-directory->hash [linklet-directory linklet-directory?])
|
|
(and/c hash? hash-eq? immutable? (not/c impersonator?))]{
|
|
|
|
Extracts the content of a @tech{linklet directory} into a @tech{hash
|
|
table}.}
|
|
|
|
|
|
@defproc[(linklet-bundle? [v any/c]) boolean?]{
|
|
|
|
Returns @racket[#t] if @racket[v] is a @tech{linklet bundle},
|
|
@racket[#f] otherwise.}
|
|
|
|
|
|
@defproc[(hash->linklet-bundle [content (and/c hash? hash-eq? immutable? (not/c impersonator?))])
|
|
linklet-bundle?]{
|
|
|
|
Constructs a @tech{linklet bundle} given mappings in the form of a
|
|
@tech{hash table}. Each key of @racket[content] must be either a
|
|
symbol or a @tech{fixnum}. Values in the hash table are unconstrained,
|
|
but the intent is that they are all @tech{linklets} or values that can
|
|
be recovered from @racket[write] output by @racket[read].}
|
|
|
|
|
|
@defproc[(linklet-bundle->hash [linklet-bundle linklet-bundle?])
|
|
(and/c hash? hash-eq? immutable? (not/c impersonator?))]{
|
|
|
|
Extracts the content of a @tech{linklet bundle} into a @tech{hash
|
|
table}.}
|
|
|
|
|
|
@defproc[(instance? [v any/c]) boolean?]{
|
|
|
|
Returns @racket[#t] if @racket[v] is a @tech{linklet instance},
|
|
@racket[#f] otherwise.}
|
|
|
|
|
|
@defproc[(make-instance [name any/c]
|
|
[data any/c #f]
|
|
[mode (or/c #f 'constant 'consistent) #f]
|
|
[variable-name symbol?]
|
|
[variable-value any/c] ... ...)
|
|
instance?]{
|
|
|
|
Constructs a @tech{linklet instance} directly. Besides associating an
|
|
arbitrary @racket[name] and @racket[data] value to the instance, the
|
|
instance is populated with variables as specified by
|
|
@racket[variable-name] and @racket[variable-value].
|
|
|
|
The optional @racket[data] and @racket[mode] arguments must be
|
|
provided if any @racket[variable-name] and @racket[variable-value]
|
|
arguments are provided. The @racket[mode] argument is used as in
|
|
@racket[instance-set-variable-value!] for every
|
|
@racket[variable-name].}
|
|
|
|
|
|
@defproc[(instance-name [instance instance?]) any/c]{
|
|
|
|
Returns the value associated to @racket[instance] as its name---either
|
|
the first value provided to @racket[make-instance] or the name of a
|
|
linklet that was instantiated to create the instance.}
|
|
|
|
|
|
@defproc[(instance-data [instance instance?]) any/c]{
|
|
|
|
Returns the value associated to @racket[instance] as its data---either
|
|
the second value provided to @racket[make-instance] or the default
|
|
@racket[#f].}
|
|
|
|
|
|
@defproc[(instance-variable-names [instance instance?]) (list symbol?)]{
|
|
|
|
Returns a list of all names for all variables accessible from
|
|
@racket[instance].}
|
|
|
|
|
|
@defproc[(instance-variable-value [instance instance?]
|
|
[name symbol?]
|
|
[fail-k any/c (lambda () (error ....))])
|
|
any]{
|
|
|
|
Returns the value of the variable exported as @racket[name] from
|
|
@racket[instance]. If no such variable is exported, then
|
|
@racket[fail-k] is used in the same way as by @racket[hash-ref].}
|
|
|
|
|
|
@defproc[(instance-set-variable-value! [instance instance?]
|
|
[name symbol?]
|
|
[v any/c]
|
|
[mode (or/c #f 'constant 'consistent) #f])
|
|
void?]{
|
|
|
|
Sets or creates the variable exported as @racket[name] in
|
|
@racket[instance] so that its value is @racket[v], as long as the
|
|
variable does not exist already as constant. If a variable for
|
|
@racket[name] exists as constant, the @exnraise[exn:fail:contract].
|
|
|
|
If @racket[mode] is a single, then the variable is created or changed
|
|
to be constant. If @racket[mode] is @racket['consistent], then
|
|
the optimizer can assume that the value has the same shape in all
|
|
instances that are used to satisfy a linklet's imports.}
|
|
|
|
|
|
@defproc[(instance-unset-variable! [instance instance?]
|
|
[name symbol?])
|
|
void?]{
|
|
|
|
Changes @racket[instance] so taht it does not export a variable as
|
|
@racket[name], as long as @racket[name] does not exist as a constant
|
|
variable. If a variable for @racket[name] exists as constant, the
|
|
@exnraise[exn:fail:contract].}
|
|
|
|
|
|
@defproc[(variable-reference->instance [varref variable-reference?]
|
|
[ref-site? any/c #f])
|
|
(if ref-site? (or/c instance? #f symbol?) instance?)]{
|
|
|
|
Extracts the instance where the variable of @racket[varref] is defined
|
|
if @var[ref-site?] is @racket[#f], and returns the instance where
|
|
@racket[varref] itself resides if @racket[ref-site?] is true. This
|
|
notion of @tech{variable reference} is the same as at the module level
|
|
and can reflect the linklet instance that implements a particular
|
|
phase of a module instance.
|
|
|
|
When @var[ref-site?] is @racket[#f], the result is @racket[#f] when
|
|
@racket[varref] is from @racket[(#%variable-reference)] with no
|
|
identifier. The result is a symbol if @racket[varref] refers to a
|
|
primitive.}
|
|
|
|
@deftogether[(
|
|
@defproc[(correlated? [v any/c]) boolean?]
|
|
@defproc[(correlated-source [stx correlated?]) any]
|
|
@defproc[(correlated-line [stx correlated?])
|
|
(or/c exact-positive-integer? #f)]
|
|
@defproc[(correlated-column [stx correlated?])
|
|
(or/c exact-nonnegative-integer? #f)]
|
|
@defproc[(correlated-position [stx correlated?])
|
|
(or/c exact-positive-integer? #f)]
|
|
@defproc[(correlated-span [stx correlated?])
|
|
(or/c exact-nonnegative-integer? #f)]
|
|
@defproc[(correlated-e [stx correlated?]) any]
|
|
@defproc[(correlated->datum [stx (or/c correlated? any/c)]) any]
|
|
@defproc[(datum->correlated [v any/c]
|
|
[srcloc (or/c correlated? #f
|
|
(list/c any/c
|
|
(or/c exact-positive-integer? #f)
|
|
(or/c exact-nonnegative-integer? #f)
|
|
(or/c exact-positive-integer? #f)
|
|
(or/c exact-nonnegative-integer? #f))
|
|
(vector/c any/c
|
|
(or/c exact-positive-integer? #f)
|
|
(or/c exact-nonnegative-integer? #f)
|
|
(or/c exact-positive-integer? #f)
|
|
(or/c exact-nonnegative-integer? #f)))
|
|
#f])
|
|
correlated?]
|
|
@defproc*[([(correlated-property [stx correlated?]
|
|
[key any/c]
|
|
[val any/c])
|
|
correlated?]
|
|
[(correlated-property [stx correlated?] [key any/c]) any/c])]
|
|
@defproc[(correlated-property-symbol-keys [stx correlated?]) list?]
|
|
)]{
|
|
|
|
Like @racket[syntax?], @racket[syntax-source], @racket[syntax-line],
|
|
@racket[syntax-column], @racket[syntax-position],
|
|
@racket[syntax-span], @racket[syntax-e], @racket[syntax->datum],
|
|
@racket[datum->syntax], @racket[syntax-property], and
|
|
@racket[syntax-property-symbol-keys], but for @tech{correlated
|
|
objects}.
|
|
|
|
Unlike @racket[datum->syntax], @racket[datum->correlated] does not
|
|
recur through the given S-expression and convert pieces to
|
|
@tech{correlated objects}. Instead, a @tech{correlated object} is
|
|
simply wrapped around the immediate value. In contrast,
|
|
@racket[correlated->datum] recurs through its argument (which is not
|
|
necessarily a @tech{correlated object}) to discover any
|
|
@tech{correlated objects} and convert them to plain S-expressions.}
|