From d5544350884ffb08893cadafe3a9f677edca092c Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Thu, 25 Oct 2012 19:28:48 -0400 Subject: [PATCH] Various edits for prose and layouts. --- index.html | 63 +++++++++++++++++++----------------- main.rkt | 94 +++++++++++++++++++++++++++++------------------------- 2 files changed, 83 insertions(+), 74 deletions(-) diff --git a/index.html b/index.html index c833dfc..08ab664 100644 --- a/index.html +++ b/index.html @@ -1,8 +1,8 @@ -Fear of Macros
►Fear of Macros
1 Introduction
2 The plan of attack
3 Transformers
3.1 What is a syntax transformer?
3.2 What is the input?
3.3 Actually transforming the input
3.4 Compile time vs. run time
4 Pattern matching: syntax-case and syntax-rules
4.1 Patterns and templates
4.1.1 "A pattern variable can’t be used outside of a template"
5 Syntax parameters
6 Robust macros: syntax-parse
7 Other questions
7.1 What’s the point of with-syntax?
7.2 What’s the point of begin-for-syntax?
7.3 What’s the point of racket/ splicing?
8 References/ Acknowledgments
9 Epilogue

Fear of Macros

Greg Hendershott

+Fear of Macros
►Fear of Macros
1 Introduction
2 The plan of attack
3 Transformers
3.1 What is a syntax transformer?
3.2 What’s the input, Kenneth?
3.3 Actually transforming the input
3.4 Compile time vs. run time
4 Pattern matching: syntax-case and syntax-rules
4.1 Patterns and templates
4.1.1 "A pattern variable can’t be used outside of a template"
5 Syntax parameters
6 Robust macros: syntax-parse
7 Other questions
7.1 What’s the point of with-syntax?
7.2 What’s the point of begin-for-syntax?
7.3 What’s the point of racket/ splicing?
8 References/ Acknowledgments
9 Epilogue

Fear of Macros

Greg Hendershott

Copyright (c) 2012 by Greg Hendershott. All rights reserved. -
Last updated 2012-10-25 18:54:58

    1 Introduction

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What is the input?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

    4 Pattern matching: syntax-case and syntax-rules

      4.1 Patterns and templates

        4.1.1 "A pattern variable can’t be used outside of a template"

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 Other questions

      7.1 What’s the point of with-syntax?

      7.2 What’s the point of begin-for-syntax?

      7.3 What’s the point of racket/splicing?

    8 References/Acknowledgments

    9 Epilogue

1 Introduction

I learned Racket after 25 years of doing C/C++ imperative programming.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first +

Last updated 2012-10-25 19:27:30

    1 Introduction

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What’s the input, Kenneth?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

    4 Pattern matching: syntax-case and syntax-rules

      4.1 Patterns and templates

        4.1.1 "A pattern variable can’t be used outside of a template"

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 Other questions

      7.1 What’s the point of with-syntax?

      7.2 What’s the point of begin-for-syntax?

      7.3 What’s the point of racket/splicing?

    8 References/Acknowledgments

    9 Epilogue

1 Introduction

I learned Racket after 25 years of doing C/C++ imperative programming.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first mind warp was functional programming. Before long I wrapped my brain around it, and went on to become comfortable and effective with many other aspects and features of Racket.

But two final frontiers remained: Macros and continuations.

I found that simple macros were easy and understandable, plus there @@ -45,8 +45,8 @@ contracts and types. These can catch mistakes and provide clear, useful error messages. It would be great if there were something similar for macros, and there is. One of the more-recent Racket macro enhancements is syntax-parse.

3 Transformers

  YOU ARE INSIDE A ROOM.

  THERE ARE KEYS ON THE GROUND.

  THERE IS A SHINY BRASS LAMP NEARBY.

  

  IF YOU GO THE WRONG WAY, YOU WILL BECOME

  HOPELESSLY LOST AND CONFUSED.

  

  > pick up the keys

  

  YOU HAVE A SYNTAX TRANSFORMER

3.1 What is a syntax transformer?

A syntax transformer is not one of the トランスフォーマ -transformers.

Instead, it is quite simple. It is a function. The function takes -syntax and returns syntax. It transforms syntax.

Here’s a transformer function that ignores its input syntax, and +transformers.

Instead, it is simply a function. The function takes syntax and +returns syntax. It transforms syntax.

Here’s a transformer function that ignores its input syntax, and always outputs syntax for a string literal:

> (define-syntax foo
    (lambda (stx)
      #'"I am foo"))
> (foo)

"I am foo"

When we use define-syntax, we’re making a transformer binding. This tells the Racket compiler, "Whenever you encounter a chunk of syntax starting with foo, please give it @@ -60,9 +60,8 @@ which is #’(displayln "hi")?

> (define-syntax (say-hi stx)
    #'(displayln "hi"))
> (say-hi)

hi

When Racket expands our program, it sees the occurrence of (say-hi), and sees it has a transformer function for that. It calls our function with the old syntax, and we return the new syntax, -which is used to evaluate and run our program.

3.2 What is the input?

Our examples so far have been ignoring the input syntax, and -outputting a fixed syntax. Usually, we want to transform the input to -something else.

But let’s start by looking at what the input is:

> (define-syntax (show-me stx)
    (print stx)
    #'(void))
> (show-me '(i am a list))

#<syntax:10:0 (show-me (quote (i am a list)))>

The (print stx) shows what our transformer is given: a syntax +which is used to evaluate and run our program.

3.2 What’s the input, Kenneth?

Our examples so far ignored the input syntax, and output a fixed +syntax. But usually we want to transform the input to something else.

Let’s start by looking closely at what the input actually is:

> (define-syntax (show-me stx)
    (print stx)
    #'(void))
> (show-me '(i am a list))

#<syntax:10:0 (show-me (quote (i am a list)))>

The (print stx) shows what our transformer is given: a syntax object.

A syntax object consists of several things. The first part is the s-expression representing the code, such as '(i am a list). Racket (and Scheme and Lisp) expressions are s-expressions— code and data have the same structure, and this makes it vastly easier @@ -72,8 +71,9 @@ information about lexical scoping (which you don’t need to worry about now, but will turn out to be important later.)

There are a variety of functions available to access a syntax object:

> (define stx #'(if x (list "true") #f))
> (syntax->datum stx)

'(if x (list "true") #f)

> (syntax-e stx)

'(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)

> (syntax->list stx)

'(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)

> (syntax-source stx)

'eval

> (syntax-line stx)

11

> (syntax-column stx)

0

When we want to transform syntax, we’ll generally take the pieces we were given, maybe rearrange their order, perhaps change some of the pieces, and often introduce brand-new pieces.

3.3 Actually transforming the input

Let’s write a transformer function that reverses the syntax it was -given:

> (define-syntax (reverse-me stx)
    (datum->syntax stx (reverse (cdr (syntax->datum stx)))))
> (reverse-me "backwards" "am" "i" values)

"i"

"am"

"backwards"

What’s going on here? First we take the input syntax, and give it to -syntax->datum. This converts the syntax into a plain old list:

> (syntax->datum #'(reverse-me "backwards" "am" "i" values))

'(reverse-me "backwards" "am" "i" values)

Using cdr slices off the first item of the list, +given:

> (define-syntax (reverse-me stx)
    (datum->syntax stx (reverse (cdr (syntax->datum stx)))))
> (reverse-me "backwards" "am" "i" values)

"i"

"am"

"backwards"

Understand Yoda, can we. Great, but how does this work?

First we take the input syntax, and give it to +syntax->datum. This converts the syntax into a plain old +list:

> (syntax->datum #'(reverse-me "backwards" "am" "i" values))

'(reverse-me "backwards" "am" "i" values)

Using cdr slices off the first item of the list, reverse-me, leaving the remainder: ("backwards" "am" "i" values). Passing that to reverse changes it to (values "i" "am" "backwards"):

> (reverse (cdr '("backwards" "am" "i" values)))

'(values "i" "am")

Finally we use syntax->datum to convert this back to @@ -82,8 +82,8 @@ compiler, and that syntax is evaluated:< described as "syntax phase vs. runtime phase". Same difference.

But a syntax transformer is run by the Racket compiler, as part of the process of parsing, expanding and understanding your code. In other words, your syntax transformer function is evaluated at compile time.

This aspect of macros lets you do things that simply aren’t possible -in normal code. One of the classic examples, is something like the -Racket if form:

(if <condition> <true-expression> <false-expression>)

If if were implemented as a function, all of the arguments +in normal code. One of the classic examples is something like the +Racket form, if:

(if <condition> <true-expression> <false-expression>)

If we implemented if as a function, all of the arguments would be evaluated before being provided to the function.

> (define (our-if condition true-expr false-expr)
    (cond [condition true-expr]
          [else false-expr]))
> (our-if #t
          "true"
          "false")

"true"

That seems to work. However, how about this:

> (define (display-and-return x)
    (displayln x)
    x)
> (our-if #t
          (display-and-return "true")
          (display-and-return "false"))

true

false

"true"

One answer is that functional programming is good, and side-effects are bad. But avoiding side-effects isn’t always practical.

Oops. Because the expressions have a side-effect, it’s obvious that @@ -150,28 +150,31 @@ transformer, and worked up from that, we won’t have that problem. We can appreciate define-syntax-rule as a convenient shorthand, but not be scared of, or confused about, that for which it’s shorthand.

4.1 Patterns and templates

Most of the materials I found for learning macros, including the -Racket Guide, do a very good job explaining how the patterns work. I’m -not going to regurgitate that here.

Instead, let’s look at some ways we’re likely to get tripped up.

4.1.1 "A pattern variable can’t be used outside of a template"

Let’s say we want to define a function with a hyphenated name, a-b, +Racket Guide, do a very good job explaining how patterns +work. I’m not going to regurgitate that here.

Instead, let’s look at some ways we’re likely to get tripped up.

4.1.1 "A pattern variable can’t be used outside of a template"

Let’s say we want to define a function with a hyphenated name, a-b, but we supply the a and b parts separately. The Racket struct -form does something like this—if we define a struct named -foo, it defines a number of functions whose names are -variations on the name foo, such as foo-field1, -foo-field2, foo?, and so on.

So let’s pretend we’re doing something like that. We want to -transform the syntax (hyphen-define a b (args) body) to -the syntax (define (a-b args) body).

A wrong first attempt is:

> (define-syntax (hyphen-define/wrong1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" a b))])
         #'(define (name args ...)
             body0 body ...))]))

eval:46:0: a: pattern variable cannot be used outside of a

template

  in: a

Huh. We have no idea what this error message means. Well, let’s see. +macro does something like this: (struct foo (field1 field2)) +automatically defines a number of functions whose names are variations +on the name foo—such as foo-field1, +foo-field2, foo?, and so on.

So let’s pretend we’re doing something like that. We want to transform +the syntax (hyphen-define a b (args) body) to the syntax +(define (a-b args) body).

A wrong first attempt is:

> (define-syntax (hyphen-define/wrong1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" a b))])
         #'(define (name args ...)
             body0 body ...))]))

eval:46:0: a: pattern variable cannot be used outside of a

template

  in: a

Huh. We have no idea what this error message means. Well, let’s see. The "template" is the #'(define (name args ...) body0 body ...) portion. The let isn’t part of that template. It sounds like we can’t use a (or b) in the -let part.

It turns out that syntax-case can have multiple -templates. The final expression is the obvious template, used to -create the output syntax. But you can use syntax a.k.a. #’ on -a pattern variable, to make a template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Our macro definition didn’t give an error, but when we tried to use -it, it didn’t work. It seems that foo-bar didn’t get defined.

This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer to work in Emacs (like I do), in a situation like this it’s -worth firing up DrRacket temporarily to use the Macro Stepper.

It shows us:

(module anonymous-module racket
  (#%module-begin
   (define-syntax (hyphen-define/wrong1.1 stx)
     (syntax-case stx ()
       [(_ a b (args ...) body0 body ...)
        (let ([name (string->symbol (format "~a-~a" #'a #'b))])
          #'(define (name args ...) body0 body ...))]))
   (define (name) #t)))

It shows that we’re expanding to (define (name) #t), but we -wanted to expand to (define (foo-bar) #t).

So the problem is we’re getting name when we wanted its -value, foo-bar.

The thing to reach for here is with-syntax. This will say -that name is in effect another pattern variable, the value of -which we want to use in our main output template.

> (define-syntax (hyphen-define/wrong1.3 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  #'a
                                                                  #'b)))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.3 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Hmm. foo-bar still not defined. Back to the Macro Stepper. It says we’re expanding to:

(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t).

Ah, that’s right. #'a and #'b are syntax +let part.

Well, syntax-case can have as many templates as you want. The +final expression is the obvious template, used to create the output +syntax. But you can use syntax (a.k.a. #’) on a pattern +variable. This makes another template, albeit a small, "fun size" +template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Our macro definition didn’t give an error, so that’s good progress! +But when we tried to use it, no luck. It seems that a function named +foo-bar wasn’t defined.

This is where the Macro Stepper in DrRacket is invaluable. Even if you +prefer to work in Emacs (like I do), this is a situation where it’s +worth using DrRacket temporarily for its Macro Stepper.

The Macro Stepper says that:

(hyphen-define/wrong1.1 foo bar () #t)

expanded to:

(define (name) #t)

We’re expanding to (define (name) #t), but we wanted to +expand to (define (foo-bar) #t).

So the problem is we’re getting name when we wanted its +value, foo-bar.

The thing to reach for here is with-syntax. This will let us +say that name is in effect another pattern variable, the +value of which we want to use in our main output template.

> (define-syntax (hyphen-define/wrong1.3 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  #'a
                                                                  #'b)))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.3 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Hmm. foo-bar is still not defined. Back to the Macro +Stepper. It says now we’re expanding to:

(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)

Ah, that’s right. #'a and #'b are syntax objects. format is printing a representation of them as syntax objects. What we want is the datum inside the syntax object, the symbols foo and bar. So we should use diff --git a/main.rkt b/main.rkt index 1a4c24b..f1624da 100644 --- a/main.rkt +++ b/main.rkt @@ -130,8 +130,8 @@ YOU HAVE A SYNTAX TRANSFORMER A syntax transformer is not one of the トランスフォーマ @hyperlink["http://en.wikipedia.org/wiki/Transformers" "transformers"]. -Instead, it is quite simple. It is a function. The function takes -syntax and returns syntax. It transforms syntax. +Instead, it is simply a function. The function takes syntax and +returns syntax. It transforms syntax. Here's a transformer function that ignores its input syntax, and always outputs syntax for a string literal: @@ -198,13 +198,12 @@ calls our function with the old syntax, and we return the new syntax, which is used to evaluate and run our program. -@subsection{What is the input?} +@subsection{What's the input, Kenneth?} -Our examples so far have been ignoring the input syntax, and -outputting a fixed syntax. Usually, we want to transform the input to -something else. +Our examples so far ignored the input syntax, and output a fixed +syntax. But usually we want to transform the input to something else. -But let's start by looking at what the input @italic{is}: +Let's start by looking closely at what the input actually @italic{is}: @i[ (define-syntax (show-me stx) @@ -254,8 +253,11 @@ given: (reverse-me "backwards" "am" "i" values) ] -What's going on here? First we take the input syntax, and give it to -@racket[syntax->datum]. This converts the syntax into a plain old list: +Understand Yoda, can we. Great, but how does this work? + +First we take the input syntax, and give it to +@racket[syntax->datum]. This converts the syntax into a plain old +list: @i[ (syntax->datum #'(reverse-me "backwards" "am" "i" values)) @@ -302,12 +304,12 @@ process of parsing, expanding and understanding your code. In other words, your syntax transformer function is evaluated at compile time. This aspect of macros lets you do things that simply aren't possible -in normal code. One of the classic examples, is something like the -Racket @racket[if] form: +in normal code. One of the classic examples is something like the +Racket form, @racket[if]: @racket[(if )] -If @racket[if] were implemented as a function, all of the arguments +If we implemented @racket[if] as a function, all of the arguments would be evaluated before being provided to the function. @i[ @@ -538,8 +540,8 @@ shorthand. @subsection{Patterns and templates} Most of the materials I found for learning macros, including the -Racket Guide, do a very good job explaining how the patterns work. I'm -not going to regurgitate that here. +Racket @italic{Guide}, do a very good job explaining how patterns +work. I'm not going to regurgitate that here. Instead, let's look at some ways we're likely to get tripped up. @@ -547,14 +549,14 @@ Instead, let's look at some ways we're likely to get tripped up. Let's say we want to define a function with a hyphenated name, a-b, but we supply the a and b parts separately. The Racket @racket[struct] -form does something like this---if we define a @racket[struct] named -@racket[foo], it defines a number of functions whose names are -variations on the name @racket[foo], such as @racket[foo-field1], +macro does something like this: @racket[(struct foo (field1 field2))] +automatically defines a number of functions whose names are variations +on the name @racket[foo]---such as @racket[foo-field1], @racket[foo-field2], @racket[foo?], and so on. -So let's pretend we're doing something like that. We want to -transform the syntax @racket[(hyphen-define a b (args) body)] to -the syntax @racket[(define (a-b args) body)]. +So let's pretend we're doing something like that. We want to transform +the syntax @racket[(hyphen-define a b (args) body)] to the syntax +@racket[(define (a-b args) body)]. A wrong first attempt is: @@ -573,10 +575,11 @@ The "template" is the @racket[#'(define (name args ...) body0 body sounds like we can't use @racket[a] (or @racket[b]) in the @racket[let] part. -It turns out that @racket[syntax-case] can have multiple -templates. The final expression is the obvious template, used to -create the output syntax. But you can use @racket[syntax] a.k.a. #' on -a pattern variable, to make a template. Let's try that: +Well, @racket[syntax-case] can have as many templates as you want. The +final expression is the obvious template, used to create the output +syntax. But you can use @racket[syntax] (a.k.a. #') on a pattern +variable. This makes another template, albeit a small, "fun size" +template. Let's try that: @i[ (define-syntax (hyphen-define/wrong1.1 stx) @@ -589,35 +592,35 @@ a pattern variable, to make a template. Let's try that: (foo-bar) ] -Our macro definition didn't give an error, but when we tried to use -it, it didn't work. It seems that foo-bar didn't get defined. +Our macro definition didn't give an error, so that's good progress! +But when we tried to use it, no luck. It seems that a function named +@racket[foo-bar] wasn't defined. This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer to work in Emacs (like I do), in a situation like this it's -worth firing up DrRacket temporarily to use the Macro Stepper. +prefer to work in Emacs (like I do), this is a situation where it's +worth using DrRacket temporarily for its Macro Stepper. -It shows us: +The Macro Stepper says that: @codeblock{ -(module anonymous-module racket - (#%module-begin - (define-syntax (hyphen-define/wrong1.1 stx) - (syntax-case stx () - [(_ a b (args ...) body0 body ...) - (let ([name (string->symbol (format "~a-~a" #'a #'b))]) - #'(define (name args ...) body0 body ...))])) - (define (name) #t))) +(hyphen-define/wrong1.1 foo bar () #t) } -It shows that we're expanding to @racket[(define (name) #t)], but we -wanted to expand to @racket[(define (foo-bar) #t)]. +expanded to: + +@codeblock{ + (define (name) #t) +} + +We're expanding to @racket[(define (name) #t)], but we wanted to +expand to @racket[(define (foo-bar) #t)]. So the problem is we're getting @racket[name] when we wanted its value, @racket[foo-bar]. -The thing to reach for here is @racket[with-syntax]. This will say -that @racket[name] is in effect another pattern variable, the value of -which we want to use in our main output template. +The thing to reach for here is @racket[with-syntax]. This will let us +say that @racket[name] is in effect another pattern variable, the +value of which we want to use in our main output template. @i[ (define-syntax (hyphen-define/wrong1.3 stx) @@ -633,9 +636,12 @@ which we want to use in our main output template. (foo-bar) ] -Hmm. @racket[foo-bar] still not defined. Back to the Macro Stepper. It says we're expanding to: +Hmm. @racket[foo-bar] is @italic{still} not defined. Back to the Macro +Stepper. It says now we're expanding to: -@racket[(define (|#-#|) #t)]. +@codeblock{ +(define (|#-#|) #t) +} Ah, that's right. @racket[#'a] and @racket[#'b] are syntax objects. @racket[format] is printing a representation of them as syntax