From d945843738bf9dab0abbdbf2caeeb25ae24ab56a Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Mon, 29 Oct 2012 11:37:10 -0400 Subject: [PATCH 01/10] Misc edits. --- main.rkt | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/main.rkt b/main.rkt index acaa56f..d437d37 100644 --- a/main.rkt +++ b/main.rkt @@ -237,10 +237,10 @@ The @racket[(print stx)] shows what our transformer is given: a syntax object. A syntax object consists of several things. The first part is the -s-expression representing the code, such as @racket['(i am a -list)]. Racket (and Scheme and Lisp) expressions are s-expressions--- -code and data have the same structure, and this makes it vastly easier -to rewrite syntax, i.e. write macros. +s-expression representing the code, such as @racket['(+ 1 +2)]. Racket (and Scheme and Lisp) expressions are s-expressions---code +and data have the same structure, and this makes it vastly easier to +rewrite syntax, i.e. write macros. Racket syntax is also decorated with some interesting information such as the source file, line number, and column. Finally, it has @@ -690,17 +690,21 @@ size" template. Let's try that: (let ([name (string->symbol (format "~a-~a" #'a #'b))]) #'(define (name args ...) body0 body ...))])) +] + +No more error---good! Let's try to use it: + +@i[ (hyphen-define/wrong1.1 foo bar () #t) (foo-bar) ] -Our macro definition didn't give an error, so that's good progress! -But when we tried to use it, no luck. It seems that a function named -@racket[foo-bar] wasn't defined. +It seems we're defining a function with a name other than +@racket[foo-bar]? This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer to work mostly in Emacs (like I do), this is a situation where -it's worth using DrRacket temporarily for its Macro Stepper. +prefer mostly to use Emacs, this is a situation where it's worth using +DrRacket at least temporarily for its Macro Stepper. @image[#:scale 0.5 "macro-stepper.png"] From bbed76d1e970d2d2400565a5abd0fc083af2f702 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 08:17:58 -0400 Subject: [PATCH 02/10] Progress on issue #1. --- main.rkt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/main.rkt b/main.rkt index d437d37..d02db99 100644 --- a/main.rkt +++ b/main.rkt @@ -531,10 +531,10 @@ definition of the helper function(s) inside @racket[begin-for-syntax]: @racketblock[ (begin-for-syntax (define (my-helper-function ....) - ....) - (define-syntax (macro-using-my-helper-function stx) - (my-helper-function ....) ....)) +(define-syntax (macro-using-my-helper-function stx) + (my-helper-function ....) + ....) ] To review: @@ -580,7 +580,7 @@ possible but tedious using list accessors such as @racket[match] to do pattern-matching. @margin-note{Historically, @racket[syntax-case] and -@racket[syntax-parse] pattern matching came first. @racket[match] was +@racket[syntax-rules] pattern matching came first. @racket[match] was added to Racket later.} It turns out that pattern-matching was one of the first improvements From defe6b1682583cee381b2d06480c7ec9f49924c6 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 08:20:39 -0400 Subject: [PATCH 03/10] Change s-expr example and delete editorializing. --- main.rkt | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/main.rkt b/main.rkt index d02db99..422701f 100644 --- a/main.rkt +++ b/main.rkt @@ -237,10 +237,7 @@ The @racket[(print stx)] shows what our transformer is given: a syntax object. A syntax object consists of several things. The first part is the -s-expression representing the code, such as @racket['(+ 1 -2)]. Racket (and Scheme and Lisp) expressions are s-expressions---code -and data have the same structure, and this makes it vastly easier to -rewrite syntax, i.e. write macros. +s-expression representing the code, such as @racket['(+ 1 2)]. Racket syntax is also decorated with some interesting information such as the source file, line number, and column. Finally, it has From 3d01e722912ec99aa86285a9b79cf05b49952a9f Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 08:21:35 -0400 Subject: [PATCH 04/10] Add reference to Ch. 8 of Dybvig. --- main.rkt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/main.rkt b/main.rkt index 422701f..35c900c 100644 --- a/main.rkt +++ b/main.rkt @@ -971,6 +971,12 @@ Looking Hygiene"], which explains syntax-parameterize. I relied heavily on that, mostly just updating it since his post was written before PLT Scheme was renamed to Racket. +@hyperlink["http://www.scheme.com/tspl4/syntax.html#./syntax:h0" "Chapter +8"] of @italic{The Scheme Programming Language} by Kent Dybvig +explains @racket[syntax-rules] and @racket[syntax-case]. Although +more "formal" in tone, you may find it helpful to read it. You never +know which explanation or examples of something will click for you. + After initially wondering if I was asking the wrong question and conflating two different issues :), Shriram Krishnamurthi looked at an early draft and encouraged me to keep going. Sam Tobin-Hochstadt and From 3011cec6fcc5bc195ab1a162e79c4282c290c85c Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 13:57:00 -0400 Subject: [PATCH 05/10] Add reference to Matthew's "You Want it When?" paper. --- main.rkt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/main.rkt b/main.rkt index 35c900c..4a4a60f 100644 --- a/main.rkt +++ b/main.rkt @@ -971,6 +971,11 @@ Looking Hygiene"], which explains syntax-parameterize. I relied heavily on that, mostly just updating it since his post was written before PLT Scheme was renamed to Racket. +Matthew Flatt's +@hyperlink["http://www.cs.utah.edu/plt/publications/macromod.pdf" "Composable +and Compilable Macros: You Want it When?"] explains how Racket handles +compile time vs. run time. + @hyperlink["http://www.scheme.com/tspl4/syntax.html#./syntax:h0" "Chapter 8"] of @italic{The Scheme Programming Language} by Kent Dybvig explains @racket[syntax-rules] and @racket[syntax-case]. Although From 209aa0fb7e492940c0136597222b191f1338c0b5 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 14:21:27 -0400 Subject: [PATCH 06/10] Explain use of make-rename-transformer. Progress on issue #1. --- main.rkt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/main.rkt b/main.rkt index 4a4a60f..a72fd41 100644 --- a/main.rkt +++ b/main.rkt @@ -909,6 +909,10 @@ value: (aif #f (displayln it) (void)) ] +Inside the @racket[syntax-parameterize], @racket[it] acts as an alias +for @racket[tmp]. The alias behavior is created by +@racket[make-rename-transformer]. + If we try to use @racket[it] outside of an @racket[aif] form, and @racket[it] isn't otherwise defined, we get an error like we want: From 40518865c64acee12a76b1adb587167adedef13d Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 18:11:44 -0400 Subject: [PATCH 07/10] Explain that `with-syntax' ~= "define pattern variable". --- main.rkt | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/main.rkt b/main.rkt index a72fd41..de1cba3 100644 --- a/main.rkt +++ b/main.rkt @@ -726,9 +726,11 @@ Well that explains it. Instead, we wanted to expand to: Our template is using the symbol @racket[name] but we wanted its value, such as @racket[foo-bar] in this use of our macro. -A solution here is @racket[with-syntax], which lets us say that -@racket[name] is something whose value can be used in our output -template: +A solution here is @racket[with-syntax]@margin-note*{You could +consider @racket[with-syntax] to mean, "define pattern variables".}, +which lets us say that @racket[name] is something whose value can be +used in our output template. In effect, it lets us say that +@racket[name] is an additional pattern variable. @i[ (define-syntax (hyphen-define/wrong1.3 stx) From ba1b4cd243c80f0476807a05e029850c2133acc4 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 18:12:06 -0400 Subject: [PATCH 08/10] Un-italicize datum. --- main.rkt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/main.rkt b/main.rkt index de1cba3..e3901e5 100644 --- a/main.rkt +++ b/main.rkt @@ -754,9 +754,9 @@ Stepper. It says now we're expanding to: ] Oh right: @racket[#'a] and @racket[#'b] are syntax objects, and -@racket[format] is printing them as such. Instead we want the datum -inside the syntax object, the symbol @racket[foo] and -@racket[bar]. To get that, we use @racket[syntax->datum]: +@racket[format] is printing them as such. Instead we want the datum in +the syntax objects (the symbols @racket[foo] and @racket[bar]). Let's +use @racket[syntax->datum]: @i[ (define-syntax (hyphen-define/ok1 stx) From c951734546dda6ad8edf1f6c2fd7360c664b6a67 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 23:52:43 -0400 Subject: [PATCH 09/10] Explain define-for-syntax --- main.rkt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/main.rkt b/main.rkt index e3901e5..f3b720a 100644 --- a/main.rkt +++ b/main.rkt @@ -534,6 +534,17 @@ definition of the helper function(s) inside @racket[begin-for-syntax]: ....) ] +In the simple case, we can also use @racket[define-for-syntax], which +composes @racket[begin-for-syntax] and @racket[define]: + +@racketblock[ +(define-for-syntax (my-helper-function ....) + ....) +(define-syntax (macro-using-my-helper-function stx) + (my-helper-function ....) + ....) +] + To review: @itemize[ From 9477c7a6d89654aaa6692500574d556cc2a943c9 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Tue, 30 Oct 2012 23:53:38 -0400 Subject: [PATCH 10/10] Regen html --- index.html | 47 ++++++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/index.html b/index.html index 830e353..a00d7f8 100644 --- a/index.html +++ b/index.html @@ -2,7 +2,7 @@ Fear of Macros

Fear of Macros

Copyright (c) 2012 by Greg Hendershott. All rights reserved. -
Last updated 2012-10-28 10:46:59

1 Preface

I learned Racket after 25 years of mostly using C and C++.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first +

Last updated 2012-10-30 23:52:57

1 Preface

I learned Racket after 25 years of mostly using C and C++.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first mind warp was functional programming. Before long I wrapped my brain around it, and went on to become comfortable and effective with many other aspects and features of Racket.

But two final frontiers remained: Macros and continuations.

I found that simple macros were easy and understandable, plus there @@ -65,9 +65,7 @@ which is used to evaluate and run our program.

3.2 

Let’s start by looking closely at what the input actually is:

> (define-syntax (show-me stx)
    (print stx)
    #'(void))
> (show-me '(i am a list))

#<syntax:10:0 (show-me (quote (i am a list)))>

The (print stx) shows what our transformer is given: a syntax object.

A syntax object consists of several things. The first part is the -s-expression representing the code, such as '(i am a list). Racket (and Scheme and Lisp) expressions are s-expressions— -code and data have the same structure, and this makes it vastly easier -to rewrite syntax, i.e. write macros.

Racket syntax is also decorated with some interesting information such +s-expression representing the code, such as '(+ 1 2).

Racket syntax is also decorated with some interesting information such as the source file, line number, and column. Finally, it has information about lexical scoping (which you don’t need to worry about now, but will turn out to be important later.)

There are a variety of functions available to access a syntax object. @@ -135,7 +133,8 @@ macro? One way to do that is put it in another module, and the racket/match module.

If instead we want to put the helper in the same module, we can’t simply define it and use it—the definition would exist at run time, but we need it at compile time. The answer is to put the -definition of the helper function(s) inside begin-for-syntax:

(begin-for-syntax
 (define (my-helper-function ....)
   ....)
 (define-syntax (macro-using-my-helper-function stx)
   (my-helper-function ....)
   ....))

To review:

  • Syntax transformers work at compile time, not run time. The good +definition of the helper function(s) inside begin-for-syntax:

    (begin-for-syntax
     (define (my-helper-function ....)
       ....))
    (define-syntax (macro-using-my-helper-function stx)
      (my-helper-function ....)
      ....)

    In the simple case, we can also use define-for-syntax, which +composes begin-for-syntax and define:

    (define-for-syntax (my-helper-function ....)
      ....)
    (define-syntax (macro-using-my-helper-function stx)
      (my-helper-function ....)
      ....)

    To review:

    • Syntax transformers work at compile time, not run time. The good news is this means we can do things like rearrange the pieces of syntax without evaluating them. We can implement forms like if that simply couldn’t work properly as run time functions.

    • More good news is that there isn’t some special, weird language @@ -152,7 +151,7 @@ rearranging the pieces into something else. As we saw, this is possible but tedious using list accessors such as cadddr. It’s more convenient and less error-prone to use match to do pattern-matching.

      Historically, syntax-case and -syntax-parse pattern matching came first. match was +syntax-rules pattern matching came first. match was added to Racket later.

      It turns out that pattern-matching was one of the first improvements to be added to the Racket macro system. It’s called syntax-case, and has a shorthand for simple situations called @@ -189,18 +188,19 @@ work it out. The "template" the error message refers to is the want. The obvious, required template is the final expression supplying the output syntax. But you can use syntax (a.k.a. #’) on a pattern variable. This makes another template, albeit a small, "fun -size" template. Let’s try that:

      > (define-syntax (hyphen-define/wrong1.1 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (let ([name (string->symbol (format "~a-~a" #'a #'b))])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/wrong1.1 foo bar () #t)
      > (foo-bar)

      foo-bar: undefined;

       cannot reference an identifier before its definition

        in module: 'program

      Our macro definition didn’t give an error, so that’s good progress! -But when we tried to use it, no luck. It seems that a function named -foo-bar wasn’t defined.

      This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer to work mostly in Emacs (like I do), this is a situation where -it’s worth using DrRacket temporarily for its Macro Stepper.

      The Macro Stepper says that the use of our macro:

      (hyphen-define/wrong1.1 foo bar () #t)

      expanded to:

      (define (name) #t)

      Well that explains it. Instead, we wanted to expand to:

      (define (foo-bar) #t)

      Our template is using the symbol name but we wanted its -value, such as foo-bar in this use of our macro.

      A solution here is with-syntax, which lets us say that -name is something whose value can be used in our output -template:

      > (define-syntax (hyphen-define/wrong1.3 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (with-syntax ([name (datum->syntax stx
                                                (string->symbol (format "~a-~a"
                                                                        #'a
                                                                        #'b)))])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/wrong1.3 foo bar () #t)
      > (foo-bar)

      foo-bar: undefined;

       cannot reference an identifier before its definition

        in module: 'program

      Hmm. foo-bar is still not defined. Back to the Macro +size" template. Let’s try that:

      > (define-syntax (hyphen-define/wrong1.1 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (let ([name (string->symbol (format "~a-~a" #'a #'b))])
               #'(define (name args ...)
                   body0 body ...))]))

      No more error—good! Let’s try to use it:

      > (hyphen-define/wrong1.1 foo bar () #t)
      > (foo-bar)

      foo-bar: undefined;

       cannot reference an identifier before its definition

        in module: 'program

      It seems we’re defining a function with a name other than +foo-bar?

      This is where the Macro Stepper in DrRacket is invaluable. Even if you +prefer mostly to use Emacs, this is a situation where it’s worth using +DrRacket at least temporarily for its Macro Stepper.

      The Macro Stepper says that the use of our macro:

      (hyphen-define/wrong1.1 foo bar () #t)

      expanded to:

      (define (name) #t)

      Well that explains it. Instead, we wanted to expand to:

      (define (foo-bar) #t)

      Our template is using the symbol name but we wanted its +value, such as foo-bar in this use of our macro.

      A solution here is with-syntaxYou could +consider with-syntax to mean, "define pattern variables"., +which lets us say that name is something whose value can be +used in our output template. In effect, it lets us say that +name is an additional pattern variable.

      > (define-syntax (hyphen-define/wrong1.3 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (with-syntax ([name (datum->syntax stx
                                                (string->symbol (format "~a-~a"
                                                                        #'a
                                                                        #'b)))])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/wrong1.3 foo bar () #t)
      > (foo-bar)

      foo-bar: undefined;

       cannot reference an identifier before its definition

        in module: 'program

      Hmm. foo-bar is still not defined. Back to the Macro Stepper. It says now we’re expanding to:

      (define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)

      Oh right: #'a and #'b are syntax objects, and -format is printing them as such. Instead we want the datum -inside the syntax object, the symbol foo and -bar. To get that, we use syntax->datum:

      > (define-syntax (hyphen-define/ok1 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (with-syntax ([name (datum->syntax stx
                                                (string->symbol (format "~a-~a"
                                                                        (syntax->datum #'a)
                                                                        (syntax->datum #'b))))])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/ok1 foo bar () #t)
      > (foo-bar)

      #t

      And now it works!

      By the way, there is a utility function in racket/syntax +format is printing them as such. Instead we want the datum in +the syntax objects (the symbols foo and bar). Let’s +use syntax->datum:

      > (define-syntax (hyphen-define/ok1 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (with-syntax ([name (datum->syntax stx
                                                (string->symbol (format "~a-~a"
                                                                        (syntax->datum #'a)
                                                                        (syntax->datum #'b))))])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/ok1 foo bar () #t)
      > (foo-bar)

      #t

      And now it works!

      By the way, there is a utility function in racket/syntax called format-id that lets us format identifier names more succinctly. As we’ve learned, we need to require the module using for-syntax, since we need it at compile time:

      > (require (for-syntax racket/syntax))
      > (define-syntax (hyphen-define/ok2 stx)
          (syntax-case stx ()
            [(_ a b (args ...) body0 body ...)
             (with-syntax ([name (format-id stx "~a-~a" #'a #'b)])
               #'(define (name args ...)
                   body0 body ...))]))
      > (hyphen-define/ok2 bar baz () #t)
      > (bar-baz)

      #t

      Using format-id is convenient as it handles the tedium of @@ -227,7 +227,9 @@ to introduce a magic variable on purpose—such as
      > (define current-foo (make-parameter "some default value"))
      > (current-foo)

      "some default value"

      > (parameterize ([current-foo "I have a new value, for now"])
          (current-foo))

      "I have a new value, for now"

      > (current-foo)

      "some default value"

      That’s a normal parameter. The syntax variation works similarly. The idea is that we’ll define it to mean an error by default. Only inside of our aif will it have a meaningful -value:

      > (require racket/stxparam)
      > (define-syntax-parameter it
          (lambda (stx)
            (raise-syntax-error (syntax-e stx) "can only be used inside aif")))
      > (define-syntax-rule (aif condition true-expr false-expr)
          (let ([tmp condition])
            (if tmp
                (syntax-parameterize ([it (make-rename-transformer #'tmp)])
                  true-expr)
                false-expr)))
      > (aif 10 (displayln it) (void))

      10

      > (aif #f (displayln it) (void))

      If we try to use it outside of an aif form, and +value:

      > (require racket/stxparam)
      > (define-syntax-parameter it
          (lambda (stx)
            (raise-syntax-error (syntax-e stx) "can only be used inside aif")))
      > (define-syntax-rule (aif condition true-expr false-expr)
          (let ([tmp condition])
            (if tmp
                (syntax-parameterize ([it (make-rename-transformer #'tmp)])
                  true-expr)
                false-expr)))
      > (aif 10 (displayln it) (void))

      10

      > (aif #f (displayln it) (void))

      Inside the syntax-parameterize, it acts as an alias +for tmp. The alias behavior is created by +make-rename-transformer.

      If we try to use it outside of an aif form, and it isn’t otherwise defined, we get an error like we want:

      > (displayln it)

      it: can only be used inside aif

      But we can still define it as a normal variable:

      > (define it 10)
      > it

      10

      6 Robust macros: syntax-parse

      TO-DO. TO-DO. TO-DO.

      7 Other questions

      Hopefully I will answer these in the course of writing the other @@ -243,7 +245,14 @@ at all, except maybe that macros seem scary and daunting.

      Eli wrote anothe Dirty Looking Hygiene, which explains syntax-parameterize. I relied heavily on that, mostly just updating it since his post was written -before PLT Scheme was renamed to Racket.

      After initially wondering if I was asking the wrong question and +before PLT Scheme was renamed to Racket.

      Matthew Flatt’s +Composable +and Compilable Macros: You Want it When? explains how Racket handles +compile time vs. run time.

      Chapter +8 of The Scheme Programming Language by Kent Dybvig +explains syntax-rules and syntax-case. Although +more "formal" in tone, you may find it helpful to read it. You never +know which explanation or examples of something will click for you.

      After initially wondering if I was asking the wrong question and conflating two different issues :), Shriram Krishnamurthi looked at an early draft and encouraged me to keep going. Sam Tobin-Hochstadt and Robby Findler also encouraged me. Matthew Flatt showed me how to make