This file summarizes certain aspects of the operation of the stepper. The Stepper's use of syntax-property: In order to make macro-unwinding possible, the stepper adds syntax-properties to elaborated code which then informs the reconstruction process about how certain macros were originally formed. In fact, the truth is slightly more complicated, because there are two rounds of macro-expansion for a program written in a teaching language. First, the program is expanded according to the macro definitions contained in the "lang" collection. Then, the code is expanded according to the macro definitions built into mzscheme. So, for instance, a beginner 'cond' expands into a mzscheme 'cond' which expands into a nested series of 'if's. Correspondingly, the stepper's addition of syntax properties is broken into two parts; those added for the beginner macros, and those added for the mzscheme macros. However, since it is harder to alter the macro expansion which occurs in mzscheme, the latter set are added not during the actual macro expansion in mzscheme but as part of a pass over the code that occurs in the annotator between expansion and the main annotation. This procedure is called 'top-level-rewrite'. Therefore, the stepper's syntax-property additions occur in two textual locations: collects/lang/private/teach.ss, and collects/stepper/private/annotate.ss (with a few stray ones in collects/lang/private/teachhelp.ss). Also, in order to be visible to the reconstructor, these properties must be explicitly transferred from the annotated source code syntax to the reconstructed syntax. This is accomplished by the 'attach-info' procedure in reconstruct.ss. Note that in the process, these properties are given names that have "user-" prepended to them. This now seems like a strange decision. Finally, the astute reader may wonder why I need to add things like 'comes-from-or, since the expression that results from the expansion of 'or' has a corresponding 'or' element in its 'origin field. The reason is that this element appears only on the outermost expression in the expansion, and during runtime reconstruction, this outermost expression may be gone. Here are the syntax properties added, the values they may be associated with, their meanings, whether they're transferred by attach-info, and where they're used: stepper-skip-completely : [ #t ] : this code should not be annotated at all. In general, this means it therefore be invisible to the reconstructor. (Not transferred.) Actually, the code is wrapped with a dummy mark, just to make sure that other marks get properly obliterated. Note that skipping an expression can turn a sensible expression into a nonsensical one. This is where the 'expressions are lists of subexpressions' abstraction and the 'expressions are one of A...Z' abstraction part ways. Uses: - applied to the check of the test-variable in the expansion of the 'or' macro. - applied to the expansion of the primitives defined with 'define-primitive'. - applied to the 'make-generative-lambda' inserted by teach.ss stepper-hint : this is the most generally applied syntax property. In general, it informs the reconstructor what macro this source expression came from. See notes below about 'being careful with stepper-hint'. (Transferred.) [ 'comes-from-cond ] : similarly, this expression came from a use of 'cond'. This tag is applied to all the if's, but not to the else clause. [ 'comes-from-let* ] : expression was expanded from let* [ 'comes-from-local ] : expression was expanded from local [ 'from-xml-box ] : expression was expanded from an xml box [ 'from-scheme-box ] : expression was expanded from a scheme box (inside an xml box) [ 'from-splice-box ] : expression was expanded from a scheme splice box (inside an xml box) [ 'comes-from-recur ] : expression was expanded from a 'recur' stepper-define-type: this is attached to the right-hand sides of defines to indicate what kind of define they came from. [ 'shortened-proc-define ] : this lambda arose from the expansion of (define (id arg ...) body). N.B.: anything tagged with this property must also have a 'stepper-proc-define-name property. [ 'lambda-define ] : this lambda arose from the expansion of (define id (lambda ( (Transferred.) Question 1: why the right-hand side? Why not on the define itself? A: because for certain kinds of define (namely those in a local), the define itself evaporates into an internal define which is then expanded in a mzscheme-native expansion into a letrec. Question 2: won't the right-hand side change, losing the mark? A: yes it will, but not for definitions of lambdas. Since there are three cases we need to distinguish, and two of them are definitions of lambdas, we can distinguish the third case by its lack of a stepper-define-type mark. Question 3: what if, through reduction, a different expression than the original one shows up ... which _does_ have one of the two stepper-define-type marks? A: to prevent this, there's another syntax-property, 'stepper-proc-define-name, which indicates the identifier to which this procedure was originally bound. In the absence of set!, this solves the problem. stepper-xml-hint : indicates whether this expression came from an xml-box. The protocol is as follows: [ 'from-xml-box ] : attached to the outer application that wraps the result of expanding an xml-box [ 'from-scheme-box ] : attached to the (begin ...) that wraps the result of expanding a scheme-box [ 'from-splice-box ] : attached to the (qq-append ...) that wraps the result of expanding a splice-box (Transferred.) stepper-xml-value-hint : like the stepper-xml-hint, used to indicate values that are the result of evaluating xml boxes. The reason this cannot be the same as the stepper-xml-hint property is that the stepper-xml-hint property is one of the "attached" ones which follows the source syntax around. This means that instances of this property used to signal values that are the result of xml-box evaluation get stomped by the stepper-xml-hint values attached to the source. [ 'from-xml-box ] : this value is the result of evaluating an xml box. stepper-proc-define-name : stores the name to which a procedure defined with the (define (fn arg ...) body) was bound. (Transferred.) stepper-orig-name: attached to an uninterned symbol used by the expansion of define so that the stepper can compare the name attached to a lambda to the name of a definition. (Not transferred.) stepper-prim-name: this is attached to the expansion of primitives introduced with the 'define-primitive' form. Its value indicates what name to give to the source term at reconstruction time. stepper-binding-type : this is actually unrelated to macro expansion. It differentiates between the various kinds of lexical bindings. (Not transferred.) [ 'macro-bound ] : this variable's binding was inserted by a macro [ 'let-bound ] : this variable's binding was in a let/*/rec [ 'lambda-bound ] : this variable's binding was in a lambda stepper-and/or-clauses-consumed : indicates the number of clauses to the left of the one associated with a given 'if' in the expansion of an 'and' or 'or'. This allows the stepper to reconstruct a partially evaluated 'and' or 'or' with the right number of 'true's or 'false's in front. (Transferred.) stepper-skipto : this instructs the annotator to look inside the current expression along a given path for the expression to be annotated. In particular, the value bound to stepper-skipto must be a list whose elements are car, cdr, or syntax-e. (Not transferred) Some uses: - applied to the 'check-undefined' check added on local-bound variables. - applied to the spurious 'let' that's wrapped around a local. Where it's used: the stepper-skipto label is used by the 2nd-pass macro-labeler and the annotator. Both are in annotate.ss. In addition to skipping inward, a stepper hint stepper-else : [ #t ] : Initially applied to the 'true' that the cond macro replaces a beginner's 'else' with, it is later transferred to the 'if' wrapping that 'true'. This is because there is no place to annotated the cond that is passed to mzscheme's expander which will result in an annotation on the 'if.' Note that it cannot be applied simply to the 'true' itself, because then, when you reduce to the then, the mark on the test (that is, the (#%datum . true)) is gone. (Transferred.) stepper-define-struct-hint : this expression came from a define-struct. The value associated with this label is the syntax-object representing the define-struct itself. stepper-test-suite-hint : this expression was the expression being tested in a test-suite-tool test. this hint indicates to the annotator that the expression should be annotated, even though it's not in one of the expected top-level shapes. STEPPER-HINT COLLISIONS The major concern with the stepper-hint is that two of them may collide. My claim that this never happens is founded on the following reasoning, KNOWN NOT TO BE RIGOROUS. 0) Tags are attached only to expressions that are produced fresh as the result of a macro expansion. (For instance, the 'if' that is the result of the expansion of 'and'.) 1) except in the circumstances listed below, the expressions with tags attached to them are not themselves macro invocations. Q.E.D. exceptions: 1) the beginner cond expands into the mzscheme cond, which is then annotated by the top-level-rewrite. In this case, the expansion of the beginner cond does not add any stepper-hint properties, so no collision can occur. 2) the intermediate let* expands into the beginner let; both add a stepper-hint. To resolve this, the expansion of let allows an existing stepper-hint to stand.