(#|
_Zodiac_
--------

Using _Zodiac_
==============

The top-level way:

   (require-library "invoke.ss" "zodiac")
   ; binds global names prefixed with `zodiac:';
   ; zodiac:internal-error and zodiac:static-error
   ;  can be redefined afterwards.

The unit/sig way:

  Elaboration time:
   (require-library "zsigs.ss" "zodiac")
   (require-library "sigs.ss" "zodiac")

  Link time:
   (require-library-unit/sig "link.ss" "zodiac")
   (require-library-unit/sig "link2.ss" "zodiac") ; see "Error Handlers" below
   Imports:
      zodiac:interface^ ; see "Error Handlers" below
      mzlib:pretty-print^
      mzlib:file^
   Exports:
      zodiac:system^ ; no `zodiac:' prefix

Reader Procedures
-----------------

> (zodiac:read p (zodiac:make-location 1 1 0 filename)) - reads from
  the port `p', which represents the file indicated by the `filename'
  string.  Returns a PROCEDURE that gets each expression as a zodiac
  AST. When the reader encounters an eof-of-file, it returns an
  instance of zodiac:eof.

Expander Procedures
-------------------

> (zodiac:scheme-expand expr [attr 'previous] [vocab #f]) - expands
  one expression, reprsented as a zodiac AST, returning a zodiac AST.

> (zodiac:scheme-expand-program exprs [attr 'previous] [vocab #f]) -
  expands several expressions, reprsented as a list of zodiac ASTs,
  returning a list of zodiac ASTs.

Zodiac AST -> S-Expression
--------------------------

> (zodiac:parsed->raw expr) - converts a zodiac AST to an S-expression
   (losing location information, obviously).

Vocabularies
------------

> beginner-vocabulary
> intermediate-vocabulary
> advanced-vocabulary
> full-vocabulary - advanced + units and objects
> scheme-vocabulary - MzScheme (unlike full-vocabulary, local, send*,
                     etc. are not present until the correcponding
                     `define-macro' expression in MzLib is evaluated
                     at elaboration time)

Handler Parameters
------------------

> (elaboration-evaluator [proc]) - parameter for the evaluatotr used
  to evaluate begin-elaboration-time bodies and the RHS of a macro
  definition.

 default: (lambda (expr parsed->raw phase)
            (eval (parsed->raw expr)))

> (user-macro-body-evaluator [proc]) - parameter for the evaluator
  used to evaluate macro applications.

 default: (lambda (x . args)
            (eval `(,x ,@(map (lambda (x) `(#%quote ,x)) args))))

Error Handlers
--------------

There are two interfaces to the error handler procedures.  Programmers 
choose the one they want by using link.ss or link2.ss, as appropriate.

Zodiac relies on two error handlers that are provided by its
>     zodiac:interface^
import: 
>   internal-error - for when things go wrong in zodiac that should
                     never go wrong
>   static-error - for input errors during read or expand

Implementors of these procedures are expected to ensure that the
procedures never return.  internal-error has the same interface in
both link.ss and link2.ss:

internal-error: where fmt-spec . args
  where -- a zodiac AST
  fmt-spec -- a format-style string
  args -- arguments for the format string

  Sample implementation:

 (define (internal-error where fmt-spec . args)
    (printf "Internal error at: ~s~n" where) ; or, pull location out of `where'
    (apply error 'internal-error fmt-spec args))

static-error has two different interfaces.  In link.ss:

static-error: where fmt-spec . args
  where -- a zodiac AST
  fmt-spec -- a format-style string
  args -- arguments for the format string

  Sample implementation:

 (define (static-error where fmt-spec . args)
    (printf "Static error at: ~s~n" where) ; or, pull location out of `where'
    (apply error 'static-error fmt-spec args))

In link2.ss:

static-error: link-text link-tag source-term fmt-spec . args
  link-text -- a string reporting the major information about the
               error; typically, this will be turned into a hyperlink
               by a user interface
  link-tag -- a tag specifying the nature of the error; typically,
              this will be used by the user interface to look up a
              database and generate a URL for the hyperlink
  fmt-spec -- a format-style string for information not in link-text
  args -- arguments for the format string

  Producers of error messages assume that the information in these
  arguments will be used in the following manner:

    <link-text>: <fmt-spec-with-args-replaced>

  Implementors may use them in any way they wish, so long as they keep
  in mind that the error producer has made the above presumption.
  Producers of errors *cannot* assume that the link-tag will be used
  (since the implementor may not have access to a hypertext medium),
  and must therefore provide enough useful information in the
  link-text and fmt-spec arguments.

  Sample implementation:

 (define (static-error link-text link-tag where fmt-spec . args)
    (printf "Error at: ~s~n" where) ; or, pull location out of `where'
    (apply error 'syntax-error
      (string-append link-text ": " fmt-spec)
      args))

Example
-------

  (require-library "invoke.ss" "zodiac")
  (let ([r ((zodiac:read (open-input-string "(cons 1 null)")
                         (zodiac:make-location 1 1 0 "string")))])
    (eval (zodiac:parsed->raw (zodiac:scheme-expand r))))
  = (list 1)


Correlating Source
------------------

Quickref:

    who           how     principal to a source expression?
    ---           ---     ---------------------------------
    'source       ...     yes
    'reader       ...     yes
    'duplicate    ...     no
    'micro        expr    iff expr is principal
    'macro        expr    iff expr is principal
    'non-source   ...     no

Details:

Zodiac's start and end locations provide a client with a mapping from
fully elaborated "E-expressions" to source S-expressions. For example,
Aries relies on the E->S mapping to hilite a specific S-expression in
response to a run-time error for a particular E-expression. Certain
tools, such as DrScheme's syntax checker, require an S->E mapping,
instead. However, the inverse of the E->S relation is not a mapping,
because E->S can map many E-expressions to one S-expression, and it
can map zero E-expressions to some S-expressions. For example, (cond
[#f 5][#t 6]) expands to (if #f 5 (if #t 6)), where the `cond'
S-expression is identified as the source of both `if'
E-expressions. Other elaborations drop an S-expression entirely, such
that an S-expression has no reprentative in the final E-expression.

The `origin' field of an E-expression provides information
for approximating an S->E function by dropping E-expression elements
from the E->S domain before inverting the relation. More specifically,
the `origin' field identifies each E-expression as either the
principal representative of its source expression or not. Zodiac
guarantees that at most one E-expression is a principal expression for
each S-expression in the source.

Principal E-Expressions
- - - - - - - - - - - -

A principal E-expression is not chosen arbitrarily. In the case of
'source, 'reader, 'macro, and 'micro principals, the E-expression is
equivalent to its S-expression in the sense that it encapsulates the
entire original expression. Thus, in the elaboration from (cond [#f
5][#t 6]) to (if #f 5 (if #t 6)), the outer `if' is identified as the
principal E-expression. The inner `if' encapsulates only a part of the
original `cond' expression (and it does not encapsulate any complete
expression from the source).

Here's a more complete dissection of a slightly larger example:

  (cond [#f 5][#t (+ 3 3)]) 
    => (if #f 5 (if #t (+ 3 3)))
       ^   ^  ^ ^   ^  ^^ ^-^- 'reader
       |   |  | |   |  |`- 'source
       |   |  | |   |  `- 'source
       |   |  | |   `- 'reader 
       |   |  | `- 'non-source
       |   |  `- 'reader
       |   `- 'reader
       `- 'micro; the how field points to the `cond' expression, which
          has a source-who value of 'source

Macros/micros that expand to macros/micros produce chains of origin
records. For example, (or a b) expands to (let ([g a]) ...) which
expands to (let-values ([(g) a]) ...). The source for the final
`letrec-values' expression is 'macro; the source-how field points to
the `let' expression, whose source is also 'macro. Finally, the
source-how field for the `let' expression is the `or' expression,
which has a 'source origin.

Non-principal E-Expressions
- - - - - - - - - - - - - -

The 'duplicate who value is used when a macro/micro duplcates a source
expression in its output, such as the `loop' in `(let loop () (loop))'
=> `(letrec ([loop (lambda () (loop))]) (loop))'). All but the first
instance of the duplicated expression get a 'duplcate source-who
annotation. (The source-how field contains the original source
record.)

The 'non-source value for the who field indicates that there is no
source expression that is equivalent to the expanded expression.  In
this case, a macro or micro must have manufactured the syntax; for
example, the `this' binding intoroduced by class* -> class*/names has
source-who value 'non-source. Of course, the location field of
"non-source" syntax still matches the syntax to a particular source
expression. Similarly, the nested `if' in the expansion of `cons'
contains a manufactured `if' expression.

Error Tags
==========

These are the tags generated by Zodiac to report static-error's. 

Using the scheme primitive `read' on this file produces a list of lists of
symbols. The symbols are the kwd: and term: tags for the language
levels. There are nine elements in the outer list. The first five list the
common, beginning, intermediate, advanced, and full scheme language levels
kwd: tags, respectively, and the last four list the beginning intermediate,
advanced, and full scheme langauge levels term: tags.

kwd Tags
--------

The following tags are prefixed with "kwd:", as in,

  kwd:lambda

They correspond exclusively to forms built into the language.

  case-lambda lambda define-internal begin-internal begin begin0 if
  with-continuation-mark quote set!-values local define local-define
  define-values struct define-struct define-structure let-struct let
  let* delay time let-values let*-values letrec-values letrec or nor
  and nand recur rec cond case evcase when unless let/cc let/ec do
  fluid-let parameterize with-handlers define-macro let-macro unquote
  unquote-splicing quasiquote unit compound-unit invoke-unit
  signature-struct signature->symbols define-signature let-signature
  unit-include unit/sig compound-unit compound-unit/sig
  invoke-unit/sig unit->unit/sig define-values global-define-values
  polymorphic mrspidey:control : type: define-type define-constructor
  reference-file require-library require-relative-library
  require-library-unit require-unit require-unit/sig
  require-library-unit/sig require-relative-library-unit
  require-relative-library-unit/sig interface class-private
  class-inherit class-rename class-sequence class class* class*/names
  ivar send send* make-generic set!  define-values require-unit
  require-unit/sig require-library-unit require-library-unit/sig
  require-relative-library-unit require-relative-library-unit/sig

Pre-Parsing Tags
----------------

> read:syntax-error

  Any syntax error during the reading phase.

> scan:syntax-error

  Any syntax error during the scanning phase.

term Tags
---------

The following tags are used to denote syntactic errors while parsing
programs.

> term:internal-def-not-foll-by-expr

  Internal definition must be followed by an expression.  A sequence
  of nothing but internal definitions is invalid (since this must
  translate into the letrec family, which needs a body).

> term:duplicate-interal-def

  Each name can be defined only once internally.

> term:case/lambda-only-in-def

  At lower language levels, procedures may only be declared
  immediately within a definition.

> term:define-internal-invalid-posn

  Not at a legal internal define position.

> term:define-illegal-implicit-begin

  A definition body has multiple body terms.  This is illegal at lower 
  language levels.

> term:if-must-have-else

  At lower language levels, if's must be two-armed.

> term:quote-not-on-symbol

  At lower language levels, quote can only be used on symbols.

> term:struct-not-id

  The field names in a structure must be valid identifiers.

> term:super-struct-invalid

  Invalid super-structure declaration syntax.

> term:super-struct-not-id

  Structure name declaration not an identifier when declaring a
  super-structure.

> term:cond-else-only-in-last

  The `else' clause in a cond must be the last such clause.

> term:cond-clause-not-in-q/a-fmt

  The cond clause is not of the proper shape.

> term:cond-=>-not-foll-by-1-rcvr

  The => clause of a cond must be followed by one expression, which
  evaluates to a receiver function.

> term:signature-out-of-context

  A name, bound to a signature, is used a context where it isn't
  legal.

> term:keyword-out-of-context

  A name, bound to a keyword, is used in a context where it isn't
  legal.

> term:empty-combination

  Use of the empty combination.  Illegal at lower language levels.

> term:app-first-term-not-var

  First term after parenthesis is a complex expression, not a variable
  reference.  Illegal at lower language levels.

> term:app-first-term-lambda-bound

  First term after parenthesis is a lambda-bound identifier.  Illegal
  at lower language levels.

> term:expected-an-identifier

  Attempt to use a syntactic non-identifier in a context that expected 
  one.

> term:repeated-identifier

  Attempt to use the same identifier twice in a context that allows
  only unique uses.

> term:invalid-identifier

  Attempt to use a non-identifier in an identifier context.

> term:arglist-after-init-value-spec

  Attempt to provide arguments without initial values following
  arguments that have initial values in an argument list
  specification.

> term:arglist-after-catch-all-arg

  Attempt to provide arguments after a catch-all argument.

> term:arglist-invalid-init-value

  Attempt to provide an initial value specification in an illegal
  position.

> term:arglist-invalid-init-var-decl

  Invalid initial value specification syntax.

> term:arglist-last-arg-no-init

  Attempt to provide an initial value in the last position of an
  argument list with a catch-all argument.

> term:arglist-invalid-syntax

  Invalid argument list syntax.

> term:proc-arity->=-1

  Attempt to define a procedure with arity < 1.  Illegal at lower
  language levels.

> term:unit-double-export

  Attempt to export the same name twice from a signed unit.

> term:duplicate-signature

  Attempt to duplicately define a signature's name.

> term:unbound-sig-name

  Attempt to refer to an signature name that hasn't been bound.

> term:signature-no-sub-unit

  Attempt to refer to a sub-unit not contained in a signature.

> term:signature-no-var

  Attempt to refer to a name not contained in a signature.

> term:unit-link-unbound-tag

  Attempt to use an unbound tag in a unit linkage specification.

> term:unit-link-duplicate-tag

  Attempt to define the same link name twice.

> term:unit-link-self-import-tag

  Attempt to create a self-import in unit linkage.

> term:unit-link-path-malformed

  Invalid linkage path syntax.

> term:unit-duplicate-import

  Attempt to import the same name twice.

> term:unit-duplicate-export

  Attempt to export the same name twice.

> term:unit-import-exported

  Attempt to export a name that has been imported.

> term:unit-defined-imported

  Attempt to define an imported name.

> term:unit-redefined-import

  Attempt to re-define an imported name within a unit.

> term:unit-export-not-defined

  Attempt to export a name that has not been defined from a unit.

> term:unit-duplicate-definition

  Attempt to define the same name twice within a unit.

> term:signature-not-matching

  Attempt to match non-matching signatures.

> term:signature-struct-illegal-omit-name

  Attempt to omit an invalid name from a signature.

> term:unit-export

  Invalid unit export syntax.

> term:c-unit-linkage

  Invalid linkage clause syntax.

> term:c-unit-export

  Invalid export clause syntax.

> term:c-unit-not-import

  Use of a non-imported identifier in a compound-unit linkage.

> term:c-unit-invalid-tag

  The use of a tag in a compound-unit linkage that is not
  syntactically correct.

> term:signature-invalid-struct-omit

  An invalid structure omission specification in a signature.

> term:signature-malformed-omit-clause

  An invalid omission specification in a signature.

> term:signature-malformed-open-clause

  An invalid open clause in a signature.

> term:signature-malformed-unit-clause

  An invalid unit clause in a signature.

> term:signature-ambiguous-:

  Use of : in signature ambiguous.

> term:no-unit-exports

  Attempt to specify sub-signatures in a signed unit's export.

> term:invalid-pos-symbol

  Invalid symbol expression syntax.

> term:invalid-pos-literal

  Invalid literal expression syntax.

> term:invalid-pos-list

  Invalid list expression syntax.

> term:invalid-pos-ilist

  Invalid improper list expression syntax.

> term:macro-error

  Any error during the evaluation of a macro application.

> term:invalid-ivar-decl

  Invalid instance variable declaration syntax.

> term:invalid-ivar-clause

  Invalid instance variable declaration syntax.

> term:set!-no-mutate-lambda-bound

  Attempt to mutate a lambda-bound variable.  Illegal at lower
  language levels.

> term:no-set!-inherited/renamed

  Attempt to mutate an inherited or renamed identifier in a class.

> term:unit-unbound-id

  Unbound identifier in a unit.

> term:def-not-at-top-level

  Attempted internal definition.  Illegal at lower language levels.

> term:invalid-intl-defn-posn

  Internal definition in an invalid position.

> term:cannot-bind-kwd

  Attempt to re-define a keyword, in a unit or at the top-level.

> term:no-set!-imported

  Attempt to mutate an imported identifier in a unit.

Tags and Language Levels
========================

This documents the language level at which each tag can appear.

Misc Tags
----------

These tags can appear at any language level:
|#(

  read:syntax-error
  scan:syntax-error

#|

kwd: Tags
---------

If these are inserted at some language level, they are automatically
present at all subsequent language levels.

    common:         |#

  kwd:define-macro
  kwd:let-macro

)#|  beginner:      |#(

  kwd:case-lambda
  kwd:lambda
  kwd:if
  kwd:quote
  kwd:define
  kwd:define-values
  kwd:struct
  kwd:define-struct
  kwd:or
  kwd:nor
  kwd:and
  kwd:nand
  kwd:cond
  kwd:require-library
  kwd:require-relative-library
  kwd:reference-file
  kwd:polymorphic
  kwd:mrspidey:control
  kwd::
  kwd:type:
  kwd:define-type
  kwd:define-constructor


)#|  intermediate:  |#(

  kwd:local
  kwd:define-structure
  kwd:let-struct
  kwd:let
  kwd:let*
  kwd:time
  kwd:let-values
  kwd:let*-values
  kwd:letrec-values
  kwd:letrec
  kwd:unquote
  kwd:unquote-splicing
  kwd:quasiquote


)#|  advanced:      |#(

  kwd:begin
  kwd:begin0
  kwd:set!
  kwd:set!-values
  kwd:delay
  kwd:recur
  kwd:rec
  kwd:case
  kwd:evcase
  kwd:when
  kwd:unless
  kwd:let/cc
  kwd:let/ec
  kwd:do
  kwd:fluid-let
  kwd:parameterize
  kwd:with-handlers


)#|  full scheme:   |#(

  kwd:with-continuation-mark
  kwd:unit
  kwd:compound-unit
  kwd:invoke-unit
  kwd:signature-struct
  kwd:signature->symbols
  kwd:define-signature
  kwd:let-signature
  kwd:unit-include
  kwd:unit/sig
  kwd:compound-unit
  kwd:compound-unit/sig
  kwd:invoke-unit/sig
  kwd:unit->unit/sig
  kwd:global-define-values
  kwd:require-library-unit
  kwd:require-unit
  kwd:require-unit/sig
  kwd:require-library-unit
  kwd:require-library-unit/sig
  kwd:require-relative-library-unit
  kwd:require-relative-library-unit/sig
  kwd:interface
  kwd:class-private
  kwd:class-inherit
  kwd:class-rename
  kwd:class-sequence
  kwd:class
  kwd:class*
  kwd:class*/names
  kwd:ivar
  kwd:send
  kwd:send*
  kwd:make-generic
)#|
term: Tags
---------

term tags are not automatically inherited by advanced levels, since
they sometimes designate an error corresponding to a restriction at a
certainl language level.  Thus, the tags are explicitly listed for
each level at which they occur.  Paradoxically, a tag can appear in a
more advanced level but not in a less advanced one.  This is typically
because the advanced level has introduced or activated a feature not
allowed in a lower level (where an attempt to use it might merely
result in a syntax error), and its misuse is flagged by this tag.

do not occur (fallbacks that are never fallen back to):

  invalid-pos-symbol
  invalid-pos-literal
  invalid-pos-list
  invalid-pos-ilist


     beginner:      |#(

  term:internal-def-not-foll-by-expr   ;; *
  term:duplicate-interal-def           ;; *
  term:case/lambda-only-in-def
  term:define-internal-invalid-posn    ;; *
  term:define-illegal-implicit-begin
  term:if-must-have-else
  term:quote-not-on-symbol
  term:cond-else-only-in-last
  term:cond-clause-not-in-q/a-fmt
  term:cond-=>-not-foll-by-1-rcvr
  term:keyword-out-of-context
  term:empty-combination
  term:app-first-term-not-var
  term:app-first-term-lambda-bound
  term:expected-an-identifier
  term:repeated-identifier
  term:invalid-identifier
  term:proc-arity->=-1
  term:set!-no-mutate-lambda-bound     ;; *
  term:def-not-at-top-level
  term:cannot-bind-kwd
  term:macro-error

)#|  intermediate:  |#(

  term:internal-def-not-foll-by-expr  ;; *
  term:duplicate-interal-def          ;; *
  term:define-internal-invalid-posn   ;; *
  term:define-illegal-implicit-begin
  term:if-must-have-else
  term:cond-else-only-in-last
  term:cond-clause-not-in-q/a-fmt
  term:cond-=>-not-foll-by-1-rcvr
  term:keyword-out-of-context
  term:empty-combination
  term:app-first-term-not-var
  term:app-first-term-lambda-bound
  term:expected-an-identifier
  term:repeated-identifier
  term:invalid-identifier
  term:proc-arity->=-1
  term:set!-no-mutate-lambda-bound    ;; *
  term:def-not-at-top-level
  term:cannot-bind-kwd
  term:macro-error


)#|  advanced:      |#(

  term:internal-def-not-foll-by-expr   ;; *
  term:duplicate-interal-def           ;; *
  term:define-internal-invalid-posn    ;; *
  term:struct-not-id
  term:super-struct-invalid
  term:super-struct-not-id
  term:cond-else-only-in-last
  term:cond-clause-not-in-q/a-fmt
  term:cond-=>-not-foll-by-1-rcvr
  term:keyword-out-of-context
  term:empty-combination
  term:expected-an-identifier
  term:repeated-identifier
  term:invalid-identifier
  term:def-not-at-top-level
  term:cannot-bind-kwd
  term:macro-error


)#|  full scheme:   |#(

  term:internal-def-not-foll-by-expr
  term:duplicate-interal-def
  term:define-internal-invalid-posn
  term:struct-not-id
  term:super-struct-invalid
  term:super-struct-not-id
  term:cond-else-only-in-last
  term:cond-=>-not-foll-by-1-rcvr
  term:keyword-out-of-context
  term:expected-an-identifier
  term:repeated-identifier
  term:invalid-identifier
  term:signature-out-of-context
  term:unit-double-export
  term:duplicate-signature
  term:unbound-sig-name
  term:signature-no-sub-unit
  term:signature-no-var
  term:unit-link-unbound-tag
  term:unit-link-duplicate-tag
  term:unit-link-self-import-tag
  term:unit-link-path-malformed
  term:unit-duplicate-import
  term:unit-duplicate-export
  term:unit-import-exported
  term:unit-defined-imported
  term:unit-redefined-import
  term:unit-export-not-defined
  term:unit-duplicate-definition
  term:signature-not-matching
  term:signature-struct-illegal-omit-name
  term:unit-export
  term:c-unit-linkage
  term:c-unit-export
  term:c-unit-not-import
  term:c-unit-invalid-tag
  term:signature-invalid-struct-omit
  term:signature-malformed-omit-clause
  term:signature-malformed-open-clause
  term:signature-malformed-unit-clause
  term:signature-ambiguous-:
  term:no-unit-exports
  term:no-set!-inherited/renamed
  term:no-set!-imported
  term:unit-unbound-id
  term:arglist-after-init-value-spec
  term:arglist-after-catch-all-arg
  term:arglist-invalid-init-value
  term:arglist-invalid-init-var-decl
  term:arglist-last-arg-no-init
  term:arglist-invalid-syntax
  term:invalid-ivar-decl
  term:invalid-ivar-clause
  term:invalid-intl-defn-posn
  term:cannot-bind-kwd
  term:macro-error
))