This file summarizes certain aspects of the operation of the stepper.

The Stepper's use of syntax-property:

In order to make macro-unwinding possible, the stepper adds
syntax-properties to elaborated code which then informs the
reconstruction process about how certain macros were originally
formed.

In fact, the truth is slightly more complicated, because there are
two rounds of macro-expansion for a program written in a teaching
language.  First, the program is expanded according to the
macro definitions contained in the "lang" collection.  Then, the
code is expanded according to the macro definitions built into
mzscheme.  So, for instance, a beginner 'cond' expands into a
mzscheme 'cond' which expands into a nested series of 'if's.

Correspondingly, the stepper's addition of syntax properties is broken
into two parts; those added for the beginner macros, and those added
for the mzscheme macros.  However, since it is harder to alter the
macro expansion which occurs in mzscheme, the latter set are added not
during the actual macro expansion in mzscheme but as part of a pass
over the code that occurs in the annotator between expansion and the
main annotation.  This procedure is called 'top-level-rewrite'.

Therefore, the stepper's syntax-property additions occur in two
textual locations: collects/lang/private/teach.ss, and
collects/stepper/private/annotate.ss (with a few stray ones
in collects/lang/private/teachhelp.ss).

In order to make it easy to move the stepper's syntax properties
as a group and also in order to check that only sane ones are
used, the stepper syntax properties are actually all grouped
together as a list stored under a  single syntax property.

In order to be visible to the reconstructor, these properties must
be explicitly transferred from the annotated source code syntax to the
reconstructed syntax. This is accomplished by the 'attach-info'
procedure in shared.rkt. Note that in the process, the source and
source-position properties are given names that have "user-" prepended
to them.  This now seems like a strange decision.

Finally, the astute reader may wonder why I need to add things like
'comes-from-or, since the expression that results from the expansion
of 'or' has a corresponding 'or' element in its 'origin field.  The
reason is that this element appears only on the outermost expression
in the expansion, and during runtime reconstruction, this outermost
expression may be gone.

Here are the syntax properties added, the values they may be
associated with, their meanings, and where they're used.

NB: this list of property names is specified in the code in
private/shared.rkt; if you add one or change its name, you should
do so in that code as well, or you'll get errors.

stepper-skip-completely :
 [ #t ] : this code should not be annotated at all.  In general, this
 means it therefore be invisible to the reconstructor.
 Actually, the code is wrapped with a dummy mark,
 just to make sure that other marks get properly obliterated.

 Note that skipping an expression can turn a sensible expression into
 a nonsensical one.  This is where the 'expressions are lists of
 subexpressions' abstraction and the 'expressions are one of A...Z'
 abstraction part ways.

 Uses:
 - applied to the check of the test-variable in the expansion of the
   'or' macro.
 - applied to the expansion of the primitives defined with
   'define-primitive'.
 - applied to the 'make-generative-lambda' inserted by teach.ss

stepper-hint :
 this is the most generally applied syntax property.  In general, it
 informs the reconstructor what macro this source expression came
 from.

 See notes below about 'being careful with stepper-hint'.

 [ 'comes-from-cond ] : similarly, this expression came from a use
 of 'cond'. This tag is applied to all the if's, but not to the else
 clause.

 [ 'comes-from-let* ] : expression was expanded from let*
 [ 'comes-from-local ] : expression was expanded from local

 [ 'from-xml-box ] : expression was expanded from an xml box
 [ 'from-scheme-box ] : expression was expanded from a scheme box
                        (inside an xml box)
 [ 'from-splice-box ] : expression was expanded from a scheme splice
		        box (inside an xml box)
 [ 'comes-from-recur ] : expression was expanded from a 'recur'
 [ 'comes-from-check-expect ] : expression was expanded from a 'check-expect'
 [ 'comes-from-check-within ] : ... from a 'check-within'
 [ 'comes-from-check-error ] : ... from a 'check-error'
 
stepper-define-type:
 this is attached to the right-hand sides of defines to indicate what
 kind of define they came from.

 [ 'shortened-proc-define ] : this lambda arose from the expansion
    of (define (id arg ...) body).  N.B.: anything tagged with this
    property must also have a 'stepper-proc-define-name
    property.

 [ 'lambda-define ] : this lambda arose from the expansion of
    (define id (lambda (

 Question 1: why the right-hand side?  Why not on the define itself?
 A: because for certain kinds of define (namely those in a local),
    the define itself evaporates into an internal define which is then
    expanded in a mzscheme-native expansion into a letrec.

 Question 2: won't the right-hand side change, losing the mark?
 A: yes it will, but not for definitions of lambdas.  Since there
    are three cases we need to distinguish, and two of them are
    definitions of lambdas, we can distinguish the third case
    by its lack of a stepper-define-type mark.

 Question 3: what if, through reduction, a different expression
  than the original one shows up ... which _does_ have one of the
  two stepper-define-type marks?
 A: to prevent this, there's another syntax-property,
    'stepper-proc-define-name, which indicates the identifier to which
    this procedure was originally bound.  In the absence of set!,
    this solves the problem.

stepper-xml-hint : indicates whether this expression came from an
xml-box.  The protocol is as follows:

 [ 'from-xml-box ] : attached to the outer application that wraps
   the result of expanding an xml-box

 [ 'from-scheme-box ] : attached to the (begin ...) that wraps
   the result of expanding a scheme-box

 [ 'from-splice-box ] : attached to the (qq-append ...) that wraps
   the result of expanding a splice-box

stepper-xml-value-hint : like the stepper-xml-hint, used to indicate
   values that are the result of evaluating xml boxes.  The reason
   this cannot be the same as the stepper-xml-hint property is that
   the stepper-xml-hint property is one of the "attached" ones which
   follows the source syntax around.  This means that instances of this
   property used to signal values that are the result of xml-box
   evaluation get stomped by the stepper-xml-hint values attached to the
   source.

 [ 'from-xml-box ] : this value is the result of evaluating an xml
   box.

stepper-proc-define-name : stores the name to which a procedure
   defined with the (define (fn arg ...) body) was bound.

stepper-orig-name: attached to an uninterned symbol used by
   the expansion of define so that the stepper can compare the
   name attached to a lambda to the name of a definition.

stepper-prim-name:
 this is attached to the expansion of primitives introduced with the
 'define-primitive' form. Its value indicates what name to give to
 the source term at reconstruction time.

stepper-binding-type :
 this is actually unrelated to macro expansion. It differentiates
 between the various kinds of lexical bindings.

 [ 'macro-bound ] : this variable's binding was inserted by a macro
 [ 'let-bound ] : this variable's binding was in a let/*/rec
 [ 'lambda-bound ] : this variable's binding was in a lambda
 [ 'non-lexical ] : this variable's identifier-binding is not
      'lexical (i.e., unbound, top-level, or a module binding
      

stepper-no-lifting-info :
 this label is applied to a let-bound-variable whose binding is not
 being annotated (because the annotator is skipping inward past it).
 For such a binding, no corresponding lifting variable is generated,
 and so the marks shouldn't try to capture it.

 
stepper-and/or-clauses-consumed :
 indicates the number of clauses to the left of the one associated
 with a given 'if' in the expansion of an 'and' or 'or'.
 This allows the stepper to reconstruct a partially evaluated
 'and' or 'or' with the right number of 'true's or 'false's
 in front.
 
stepper-skipto :
 this instructs the annotator to look inside the current expression
 along a given path for the expression to be annotated.  In
 particular, the value bound to stepper-skipto must be a list whose
 elements are car, cdr, or syntax-e.

Some uses:
 - applied to the 'check-undefined' check added on local-bound variables.
 - applied to the spurious 'let' that's wrapped around a local.

Where it's used: the stepper-skipto label is used by the 2nd-pass
macro-labeler and the annotator.  Both are in annotate.ss.  In addition
to skipping inward, a stepper hint

stepper-skipto/discard :

 This is like stepper-skipto, except that it makes the stepper
 replace the expression the property is attached to by the
 subexpression indicated by its value.

 (This is used in the contracts implementation for "Die Macht der
 Abstraktion", where procedures are wrapped in a contract-checking
 context that has no impact on the reduction semantics.)

stepper-replace :

  This is like stepper-skipto/discard, except that it makes the
  stepper replace the expression the property is attached to by the
  value of the property.

stepper-else :
 [ #t ] : Initially applied to the 'true' that the cond macro
  replaces a beginner's 'else' with, it is later transferred
  to the 'if' wrapping that 'true'.  This is because there is no
  place to annotated the cond that is passed to mzscheme's
  expander which will result in an annotation on the 'if.'
  Note that it cannot be applied simply to the 'true' itself,
  because then, when you reduce to the then, the mark on the
  test (that is, the (#%datum . true)) is gone.
  
stepper-black-box-expr :
  this expression should be treated as a "black box"; evaluated without
  annotation, and "transparent" in the stepper, in the sense that the
  original expression should appear in the completed expressions without
  alteration.
  
  Examples:
  - define-struct
  - require
  
stepper-test-suite-hint :
  this expression was the expression being tested in a test-suite-tool
  test.  this hint indicates to the annotator that the expression should
  be annotated, even though it's not in one of the expected top-level
  shapes.

stepper-highlight :
  this expression will be highlighted.
  (Not currently tranferred...?)

stepper-fake-exp :
  this expression does not occur directly in the source; reconstruct specially.
  used for begin.

stepper-args-of-call [ADDED BY RECONSTRUCTOR] :
  this reconstructed (...) expression is the result of a call with these args.
  used by the check-expect unwinder to figure out the expected values.

stepper-hide-completed : don't show the final top-level expression binding
  for this identifier.

stepper-hide-reduction : don't show any reductions where this term is 
  associated with the topmost mark

stepper-use-val-as-final : use the return value of this expression as a
  "completed" val in the stepper. Used for test cases.

stepper-lifted-name : used by the reconstructor to tell the unwinder
  what the lifted name of a variable is.

lazy-op : ???


STEPPER-HINT COLLISIONS

 The major concern with the stepper-hint is that two of them may
 collide.  My claim that this never happens is founded on the following
 reasoning, KNOWN NOT TO BE RIGOROUS.

 0) Tags are attached only to expressions that are produced fresh as
 the result of a macro expansion. (For instance, the 'if' that is
 the result of the expansion of 'and'.)

 1) except in the circumstances listed below, the expressions with tags
 attached to them are not themselves macro invocations.

 Q.E.D.

 exceptions:

1) the beginner cond expands into the mzscheme cond, which is
then annotated by the top-level-rewrite. In this case, the expansion
of the beginner cond does not add any stepper-hint properties, so no
collision can occur.

2) the intermediate let* expands into the beginner let; both add a
stepper-hint.  To resolve this, the expansion of let allows an existing
stepper-hint to stand.


KINDS OF BREAKPOINTS:

'normal-break : occurs before an expression's evaluation. The "before"
  steps.

'result-exp-break : occurs before an expression's evaluation that is the
  "after" of some other step. E.G., the rhs of a cond, the body of
  a procedure, etc.

'result-value-break : occurs when a value is returned. Also corresponds
  to an "after" step.

'normal-break/values : occurs before an expression's evaluation, when
  that expression has intermediate values. For instance, an "if"'s
  reduction occurs after its test value has been evaluated, so the
  "before" step for the "if" must already know about the test value.

'expr-finished-break : occurs when a top-level expression is complete.

'double-break : occurs as part of the evaluation of a let or local,
  after the evaluation of all the binding rhses. Called a double-break
  because it must generate two steps.