The optimizer should only run when the code being compiled could
directly access racket/unsafe/ops. This prevents unsoundness in Typed
Racket from giving untrusted code access to dangerous operations.
This PR adds about half of the needed primitives and logic for
reasoning about linear integer arithmetic in programs with interesting
dependent types. Things have been added in a way s.t. programs will
still continue to typecheck as they did, but if you want integer literals
and certain operations (e.g. *,+,<,<=,=,>=,>) to include linear inequality
information by default, you need to include the
'#:with-linear-integer-arithmetic' keyword at the top of your module.
The other features needed to get TR to be able to check things like
verified vector operations will be to ajust function types so
dependencies can exist between arguments and a minor tweak to get
type inference to consider the symbolic objects of functions arguments.
These features should be coming shortly in a future pull request.
Keyword functions are a little tricky. This PR addresses issues
checking the body of kw functions.
Basically, a function with keyword arguments such as inc:
(define (inc x #:n [n 1])
(+ x n))
actually expands into a more complex function with 3 arguments that
looks something resembling the following:
(define (inc-expanded n* n-given? x)
(let ([n (if n-given? n* 1)]) (+ x n)))
and calls to inc are converted to match this form:
(inc 42) => (inc-expanded #f #f 42)
(inc 42 #:n 2) => (inc-expanded 2 #t 42)
Note that each optional keyword argument has a boolean flag argument
that signals whether or not the caller provided that keyword argument.
This PR takes advantage of the observation that the value for the n*
argument in inc is only reachable in code when n-given? is #t, and so,
assuming the kw-expansion protocol always only accesses n* if n-given?
is #t, we can actually safely check the body of the function against
the following simple but correct type:
(-> Number Boolean Number Number)
An alternative previous approach expanded the function type into every
possible combination of optional argument and optional argument flag,
but this was prohibitively expensive.
An opaque object contract is stronger than another (opaque) object contract if:
- it has stronger field/method contracts on fields/methods common to both
- and it has no more field/method contracts than the other, if the other is opaque
`field` needs to be a literal; with the bug, the following syntax
```
(object/c-opaque (any-var (->m any/c)))
```
made a contract with 1 field (named `->m`, value `any/c`)
This PR primarily changes how we represent Base types and unions of
Base types. Basically, Base types now contain a bits field, which
contains an exact-nonnegative-integer? with exactly one bit set to
1. This allows us to represent unions of Base types by simply OR-ing
together the various base bits. We store these unions in a new type
called a BaseUnion (which contains a field for numeric Base type bits
and a field for non-numeric Base type bits). We can perform set
operations on BaseUnion types rather quickly (using simple bitwise
arithmetic operations).
To make Union and BaseUnion work together nicely, the Union type now
has a field for non-Base types, and a field which contains any and all
Base types placed directly in the Union ( either as a Base if there is
only one, or as a BaseUnion if there are more than one).
Other changes present in this PR:
Base types are now "closed" -- i.e. Base types are only declared in
base-types.rkt and numeric-base-types.rkt with a special macro that
assigns them their respective bits. The constructor make-Base is no
longer provided for miscellaneous usages.
Some singleton Value types were moved to Base so all of our common
unions of basic types can fit into the new BaseUnion type (namely
Null, Void, One, Zero, and the booleans).
A new Val-able match expander lets us match on singleton types that
used to all be Value types, but, as described above, now some are Base
types.
Unions contain deterministically ordered, duplicate free lists (in
addition to sets for equality and constant time membership checks), so
iterating over Unions can be done deterministically (yay!) -- this
gets rid of some otherwise problematic behavior in areas like type
inference, where the order Unions are iterated over can actually
affect the results (i.e. if two valid type inferences are possible,
nondeterministic ordering means we can sometimes get one and sometimes
get another, which makes for particularly difficult to debug issues
and in general has no immediate solution (both substitutions are
valid, after all!))
Previously expected propositions were erased frequently
(at lets and ifs) and checking for logical entailment
was unidirectional instead of bidirectional. In other words,
instead of checking if propositions held at the leaves
of the AST, we would typecheck the AST and blindly propagate
up ALL logical info we learned at each step. This meant that
we would get exponential blow up of propositions even when
we didn't care about their content.
With this commit, instead now we send down expected types
*and* propositions so we can verify expected types and
propisitions are satisfied at leaves, thereby relieving the
need to constantly report up huge amounts of logical info
while typechecking.
Moving to eager propagating of bottom works for most cases,
but in some cases flattening types such as (Pairof Bottom Any)
to Bottom made things like type inference break for some cases
(since (Listof Nothing) == Null, and (Listof A) did not structurally
like up like it used to). Perhaps w/ a little more effort
inference and any other potential issues could work better
with propagating bottom, but for now we'll be slightly less
aggressive about it.
i.e. this fixes pfds, which commit 8e7f390 broke.
Prior to this change (which was Typed Racket PR 469) all internal TR
objects (Reps) were interned and kept around for the entire duration
of type checking. Because of this, frequent operations that rebuilt
types were particularly costly (e.g. various forms of
substitution). To recoup some of this cost, caching was being used in
a lot of places. This PR sought to remove interning as the default
behavior for Reps and allow for more flexibility in how we approach
time/space performance needs going forward.
The following changes were included in this overhaul:
Interning: All Reps are no longer interned. Right now we only intern
unions and some propositions.
Rep generic operations: we now use racket/generic so we're not
reinventing this wheel.
Singletons: Reps (e.g. TrueProp, Univ, etc) can be declared singleton,
which creates a single instance of the rep that all visible operations
(even within the declaring module) are defined in terms of
(e.g. predicates are defined as (λ (x) (eq? x singleton-instance)),
etc).
Custom constructors: Custom constructors can be specified for Reps,
which allows for simple normalization, interning, or other invariants
to be enfored whenever a Rep is created.
Union: Unions used to try to ensure no obviously overlaping types
would inhabit the same Union (e.g. (U String (Pairof Any Any) (Pairof
Int Int)) would be simplified to (U String (Pairof Any Any))). This,
however, required frequent calls to subtyping every time a Union was
modified and working with Unions thus had a high cost (another thing
that caching was used to reduce). Instead of this, Unions now enforce
a much simpler set of invariants on their members: (1) No duplicates
(by virtue of using a hash-based set), (2) Any and Nothing do not
appear in unions, and (3) Nested unions are flattened. Also, using a
hashset as the internal data structure meant that we could easily
intern unions w.r.t. equal? equality. NOTE: we do reduce unions to not
contain obviously overlapping terms when printing to users and when
generating contracts (so obviously and avoidable inneficient contracts
are not generated – See union.rkt for 'normalize-type').
Subtyping changes: Subtyping has been designed to reduce dispatch time
w/ a switch since we no longer cache _all_ subtyping calls (we only
cache subtyping results for unions since they have some costly
subtyping).
prop-ops changes: AndProps now are careful to sort OrProps by length
before building the resulting proposition. This is done because
OrProp implication only checks if one Or is a subset of another Or.
By ordering Or props by size, we only ever check if an OrProp implies
another if its size is <= the other OrProp. This also makes the
smart constructor '-and' more robust, since the order the props
appear does not affect if some Ors are kept or not.
Testing: More subtype tests have been added (we are still probably
relying too much on typecheck-tests.rkt and not the more granular unit
tests in general). Also, typecheck-tests.rkt has been changed to
check for type-equivalence (i.e. subtyping in both directions)
instead of equal? equivalence.
This commit cleans up some helper functions that have been
pretty awful for a while. The code is (hopefully) more readable
and it does less work (i.e. unneeded substitutions are no longer
performed).
The previous fix relied on finding and manipulating all dead code.
But we missed some; in particular code of the form:
(begin (error 'x) ...dead...)
So switch to a different strategy that tolerates untraversed
dead code.
This is a major to some of the internal representation of things
within Typed Racket (mostly affecting structs that inherited from Rep
(see rep/rep-utils.rkt)), and lots of tweaks and bug fixes that
happened along the way.
This PR includes the following major changes:
A new rep-utils implementation, which uses struct properties for the
generic operations and properties of the various Reps (see
rep-utils.rkt)
More specific Rep inheritance (i.e. arr no longer inherits from Type,
because it is not a Type, etc ...) (see type-rep.rkt, core-rep.rkt,
values-rep.rkt), and thus things like Type/c no longer exist
New Rep's to classify the things that are no longer Type or Prop,
(such as PropSets, SomeValues, Results, etc -- see core-rep.rkt and
values-rep.rkt)
uses of type-case now replaced by uses of Rep-fold and Rep-walk
structural types can specify their fields' variance and operations
like subtyping and free-vars can generically operate over these types
(see type-rep.rkt)
type-mask replaces types key -- types masks are described in detail in
(rep/type-mask.rkt)
Types can specify a predicate to recognize their "top type" via [#:top
pred])
There is an explicit 'Bottom' type now (i.e. neither union or
intersection are used)
subtyping re-organized, slight tweaking to inference
various environments got for-each functions in addition to the map
functions they had (e.g. type-name-env.rkt)
Empty is no longer an Object? -- the OptObject? predicate checks for
either Object or Empty, and so it is easier to be clear about where
Empty makes sense appearing and where it does not
Previously signatures were created with promises in their fields, now
we create a promise around each signature (this way the contracts for
Signature fields are cleaner)
Names for structs now use the args field to describe how many type
arguments they take (Note: this could use further tidying for sure!)
simplified the propositional logic code in several places, got rid of
escape continuations, etc (see prop-ops.rkt, tc-envops.rkt,
tc-metafunctions.rkt)
we now use subsumption more to simplify type results from type
checking, e.g. if the type does not overlap w/ false, it's false
proposition is FalseProp, etc (see tc-expr-unit.rkt and prop-ops.rkt,
the function is called reduce-tc-results/subsumption)
updating along a path will now intersect with the expected structural
type if it is not encountered (e.g. updating Any with (Int @ car) now
produces (Pairof Int Any) instead of Any -- see update.rkt)
lots of tests were tweaked to match up w/ the new prop subsumption
that occurs
remove was renamed subtract (so as to not conflict w/ racket/base's
remove)
a restrict function was added, which acts like intersect but is never
additive (i.e. it will never create an intersection if it can't figure
out how the two types relate -- see intersect.rkt)
tc-subst was modified to substitute out all the variables leaving
scope at once (and I simplified/tweaked some of the logic in there a
little, see tc-subst.rkt)
Type checking function applications now propagates information learned
why type checking the arguments, (e.g. (begin (f (assert x boolean?))
...)) ; the remainder of the begin is aware that x is a boolean)
Accomodate a potential changes to `syntax-parameterize`
and the way it's use by `class`, where the changes
mostly introduce some `#%expression` wrappers.
Before this, row instantiation was done with an ad-hoc
and undocumented syntax. Adding a new form works better
because rows should not be parsed as types.
This fills the corresponding entries in the cast table with a Dead-Code
type so that when the contract-generation pass calls the contract-def
thunk, it finds that in the table.
* call compute-constraints instead of sc->constraints in get-max-contract-kind
* test cast on an intersection type involving Rec
* remove memory limit on sandboxed-unsafe-ops test
Instead of looking at the size of the type, check to see
how many times each type is referenced in other types.
Only lift a type out as a definition if it reaches a
threshold (currently set to 5 refs).
This reduces the zo size of typed/private/framework-types
by roughly 1MB (more than half).
Also move more of the env code generation into the
init-envs.rkt file itself.
Instead of storing a parameter with a box, use a
module-level variable with a hash table and a separate
queue. This separates memoization and the generation
of definitions, which is cleaner.
