From fdb06cfc11194435e7d0d06fcc4245a326ab5f83 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Fri, 26 Oct 2012 14:34:33 -0400 Subject: [PATCH] Regen HTML. --- index.html | 133 +++++++++++++++++++++++++++-------------------------- 1 file changed, 68 insertions(+), 65 deletions(-) diff --git a/index.html b/index.html index da7b61b..1ab57d0 100644 --- a/index.html +++ b/index.html @@ -1,8 +1,8 @@ -Fear of Macros
►Fear of Macros
1 Introduction
2 The plan of attack
3 Transformers
3.1 What is a syntax transformer?
3.2 What’s the input?
3.3 Actually transforming the input
3.4 Compile time vs. run time
3.5 begin-for-syntax
4 Pattern matching: syntax-case and syntax-rules
4.1 Patterns and templates
4.1.1 "A pattern variable can’t be used outside of a template"
5 Syntax parameters
6 Robust macros: syntax-parse
7 Other questions
7.1 What’s the point of with-syntax?
7.2 What’s the point of begin-for-syntax?
7.3 What’s the point of racket/ splicing?
8 References/ Acknowledgments
9 Epilogue

Fear of Macros

Greg Hendershott

+Fear of Macros
►Fear of Macros
1 Preface
2 The plan of attack
3 Transformers
3.1 What is a syntax transformer?
3.2 What’s the input?
3.3 Actually transforming the input
3.4 Compile time vs. run time
3.5 begin-for-syntax
4 Pattern matching: syntax-case and syntax-rules
4.1 Patterns and templates
4.1.1 "A pattern variable can’t be used outside of a template"
5 Syntax parameters
6 Robust macros: syntax-parse
7 Other questions
7.1 What’s the point of with-syntax?
7.2 What’s the point of begin-for-syntax?
7.3 What’s the point of racket/ splicing?
8 References and Acknowledgments
9 Epilogue

Fear of Macros

Greg Hendershott

Copyright (c) 2012 by Greg Hendershott. All rights reserved. -
Last updated 2012-10-26 13:34:38

    1 Introduction

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What’s the input?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

      3.5 begin-for-syntax

    4 Pattern matching: syntax-case and syntax-rules

      4.1 Patterns and templates

        4.1.1 "A pattern variable can’t be used outside of a template"

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 Other questions

      7.1 What’s the point of with-syntax?

      7.2 What’s the point of begin-for-syntax?

      7.3 What’s the point of racket/splicing?

    8 References/Acknowledgments

    9 Epilogue

1 Introduction

I learned Racket after 25 years of doing C/C++ imperative programming.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first +

Last updated 2012-10-26 14:32:54

    1 Preface

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What’s the input?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

      3.5 begin-for-syntax

    4 Pattern matching: syntax-case and syntax-rules

      4.1 Patterns and templates

        4.1.1 "A pattern variable can’t be used outside of a template"

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 Other questions

      7.1 What’s the point of with-syntax?

      7.2 What’s the point of begin-for-syntax?

      7.3 What’s the point of racket/splicing?

    8 References and Acknowledgments

    9 Epilogue

1 Preface

I learned Racket after 25 years of mostly using C and C++.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first mind warp was functional programming. Before long I wrapped my brain around it, and went on to become comfortable and effective with many other aspects and features of Racket.

But two final frontiers remained: Macros and continuations.

I found that simple macros were easy and understandable, plus there @@ -15,10 +15,10 @@ going on. Gah.

I’m starting to write this at the point where the sha emerging from the fog.

If you have any corrections, criticisms, complaints, or whatever, please let me know.

My primary motive is selfish. Explaining something forces me to learn -it more thoroughly. Plus I expect that if I write something with -mistakes, other people will be eager to point them out and correct -me. Is that a social-engineering variation of meta-programming? Next -question, please. :)

Finally I do hope it may help other people who have a similar +it more thoroughly. Plus if I write something with mistakes, other +people will be eager to point them out and correct me. Is that a +social-engineering variation of meta-programming? Next question, +please. :)

Finally I do hope it may help other people who have a similar background and/or learning style as me.

I want to show how Racket macro features have evolved as solutions to problems or annoyances. I learn more quickly and deeply when I discover the answer to a question I already have, or find the solution @@ -38,12 +38,12 @@ if we want to write the ever-popular anaphoric if, with a "magic variable"? It turns out we’ve been protected from making certain kind of mistakes. When we want to do this kind of thing on purpose, we use a syntax parameter. [There are other, older ways to do -this. We won’t look at them. We also won’t spend a lot of time talking -about "hygiene".]

4. Finally, we’ll realize that our macros could be smarter when -they’re used in error. Normal Racket functions can optionally have -contracts and types. These can catch mistakes and provide clear, +this. We won’t look at them. We also won’t spend a lot of time +advocating "hygiene"—we’ll just stipulate that it’s good.]

4. Finally, we’ll realize that our macros could be smarter when +they’re used in error. Normal Racket functions optionally can have +contracts and types. These catch usage mistakes and provide clear, useful error messages. It would be great if there were something -similar for macros, and there is. One of the more-recent Racket macro +similar for macro. There is. One of the more-recent Racket macro enhancements is syntax-parse.

3 Transformers

  YOU ARE INSIDE A ROOM.

  THERE ARE KEYS ON THE GROUND.

  THERE IS A SHINY BRASS LAMP NEARBY.

  

  IF YOU GO THE WRONG WAY, YOU WILL BECOME

  HOPELESSLY LOST AND CONFUSED.

  

  > pick up the keys

  

  YOU HAVE A SYNTAX TRANSFORMER

3.1 What is a syntax transformer?

A syntax transformer is not one of the トランスフォーマ transformers.

Instead, it is simply a function. The function takes syntax and returns syntax. It transforms syntax.

Here’s a transformer function that ignores its input syntax, and @@ -115,28 +115,27 @@ something like racket/match, we have to require it ourselves—and require it for compile time. Instead of using plain (require racket/match), the way to say this is to use (require (for-syntax racket/match))—the for-syntax -part meaning, "for compile time".

So let’s try that:

> (require (for-syntax racket/match))
> (define-syntax (our-if-using-match-v2 stx)
    (match (syntax->list stx)
      [(list _ condition true-expr false-expr)
       (datum->syntax stx `(cond [,condition ,true-expr]
                                 [else ,false-expr]))]))
> (our-if-using-match-v2 #t "true" "false")

"true"

3.5 begin-for-syntax

We used (require (for-syntax racket/match)) to -require match because we needed to use -match at compiler time.

What if we wanted to make some helper function that could be used by a +part meaning, "for compile time".

So let’s try that:

> (require (for-syntax racket/match))
> (define-syntax (our-if-using-match-v2 stx)
    (match (syntax->list stx)
      [(list _ condition true-expr false-expr)
       (datum->syntax stx `(cond [,condition ,true-expr]
                                 [else ,false-expr]))]))
> (our-if-using-match-v2 #t "true" "false")

"true"

3.5 begin-for-syntax

We used for-syntax to require the +racket/match module because we needed to use match +at compile time.

What if we wanted to define our own helper function to be used by a macro? One way to do that is put it in another module, and -require it using for/syntax, just like we did for -match.

If we want to put the helper in the same module, we can’t simply -define it and use it – the definition will exist at run -time, but we need it at compile time. The answer is to put the -definition of the helper function(s) inside begin-for-syntax:

(begin-for-syntax
 (define (my-helper-function x)
   (* 2 x))
 (define-syntax (my-macro stx)
   (my-helper-function ....)
   ....))

To review:

  • Syntax transformers work at compile time, not run time. The good -news is this means we can do things like delay evaluation, and -implement forms like if which simply couldn’t work properly -as run time functions.

  • Some other good news is that there isn’t some special, weird language +require it using for-syntax, just like we did with +the racket/match module.

    If instead we want to put the helper in the same module, we can’t +simply define it and use it—the definition would exist at +run time, but we need it at compile time. The answer is to put the +definition of the helper function(s) inside begin-for-syntax:

    (begin-for-syntax
     (define (my-helper-function ....)
       ....)
     (define-syntax (macro-using-my-helper-function stx)
       (my-helper-function ....)
       ....))

    To review:

    • Syntax transformers work at compile time, not run time. The good +news is this means we can do things like rearrange the pieces of +syntax without evaluating them. We can implement forms like +if that simply couldn’t work properly as run time functions.

    • More good news is that there isn’t some special, weird language for writing syntax transformers. We can write these transformer -functions using familiar Racket code.

    • The semi-bad news is that the familiarity can make it easy to forget +functions using the Racket language we already know and lovex.

    • The semi-bad news is that the familiarity can make it easy to forget that we’re not working at run time. Sometimes that’s important to remember.

      • For example only racket/base is required for us automatically. If we need other modules, we have to require them, and -do so for compile time using -(require (for-syntax)).

      • Similarly, if we define helper functions in the same file/module that -uses them, we need to nestle their definitions inside a -begin-for-syntax form. Doing so makes their definition -available at compile time.

    4 Pattern matching: syntax-case and syntax-rules

    Most useful syntax transformers work by taking some input syntax, and +do so for compile time using for-syntax.

  • Similarly, if we want to define helper functions in the same +file/module as the macros that use them, we need to wrap the +definitions inside a begin-for-syntax form. Doing so makes +them available at compile time.

4 Pattern matching: syntax-case and syntax-rules

Most useful syntax transformers work by taking some input syntax, and rearranging the pieces into something else. As we saw, this is possible but tedious using list accessors such as cadddr. It’s more convenient and less error-prone to use @@ -145,10 +144,10 @@ possible but tedious using list accessors such as added to Racket later.

It turns out that pattern-matching was one of the first improvements to be added to the Racket macro system. It’s called syntax-case, and has a shorthand for simple situations called -define-syntax-rule.

Recall our previous example:

(require (for-syntax racket/match))
(define-syntax (our-if-using-match-v2 stx)
  (match (syntax->list stx)
    [(list _ condition true-expr false-expr)
     (datum->syntax stx `(cond [,condition ,true-expr]
                               [else ,false-expr]))]))

Here’s what it looks like using syntax-case:

> (define-syntax (our-if-using-syntax-case stx)
    (syntax-case stx ()
      [(_ condition true-expr false-expr)
       #'(cond [condition true-expr]
               [else false-expr])]))
> (our-if-using-syntax-case #t "true" "false")

"true"

Pretty similar, huh? The pattern part looks almost exactly the -same. The "template" part—where we specify the new syntax—is -simpler. We don’t need to do quasi-quoting and unquoting. We don’t need -to use datum->syntax. We simply supply a template, which uses +define-syntax-rule.

Recall our previous example:

(require (for-syntax racket/match))
(define-syntax (our-if-using-match-v2 stx)
  (match (syntax->list stx)
    [(list _ condition true-expr false-expr)
     (datum->syntax stx `(cond [,condition ,true-expr]
                               [else ,false-expr]))]))

Here’s what it looks like using syntax-case:

> (define-syntax (our-if-using-syntax-case stx)
    (syntax-case stx ()
      [(_ condition true-expr false-expr)
       #'(cond [condition true-expr]
               [else false-expr])]))
> (our-if-using-syntax-case #t "true" "false")

"true"

Pretty similar, huh? The pattern matching part looks almost exactly +the same. The way we specify the new syntax is simpler. We don’t need +to do quasi-quoting and unquoting. We don’t need to use +datum->syntax. Instead, we supply a "template", which uses variables from the pattern.

There is a shorthand for simple pattern-matching cases, which expands into syntax-case. It’s called define-syntax-rule:

> (define-syntax-rule (our-if-using-syntax-rule condition true-expr false-expr)
    (cond [condition true-expr]
          [else false-expr]))
> (our-if-using-syntax-rule #t "true" "false")

"true"

Here’s the thing about define-syntax-rule. Because it’s so simple, define-syntax-rule is often the first thing people are @@ -170,18 +169,19 @@ automatically defines a number of functions whose names are variations on the name foo—such as foo-field1, foo-field2, foo?, and so on.

So let’s pretend we’re doing something like that. We want to transform the syntax (hyphen-define a b (args) body) to the syntax -(define (a-b args) body).

A wrong first attempt is:

> (define-syntax (hyphen-define/wrong1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" a b))])
         #'(define (name args ...)
             body0 body ...))]))

eval:46:0: a: pattern variable cannot be used outside of a

template

  in: a

Huh. We have no idea what this error message means. Well, let’s see. -The "template" is the #'(define (name args ...) body0 body ...) portion. The let isn’t part of that template. It -sounds like we can’t use a (or b) in the -let part.

Well, syntax-case can have as many templates as you want. The -final expression is the obvious template, used to create the output -syntax. But you can use syntax (a.k.a. #’) on a pattern -variable. This makes another template, albeit a small, "fun size" -template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Our macro definition didn’t give an error, so that’s good progress! +(define (a-b args) body).

A wrong first attempt is:

> (define-syntax (hyphen-define/wrong1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" a b))])
         #'(define (name args ...)
             body0 body ...))]))

eval:46:0: a: pattern variable cannot be used outside of a

template

  in: a

Huh. We have no idea what this error message means. Well, let’s try to +work it out. The "template" the error message refers to is the +#'(define (name args ...) body0 body ...) portion. The +let isn’t part of that template. It sounds like we can’t use +a (or b) in the let part.

In fact, syntax-case can have as many templates as you +want. The obvious, required template is the final expression supplying +the output syntax. But you can use syntax (a.k.a. #’) on a +pattern variable. This makes another template, albeit a small, "fun +size" template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Our macro definition didn’t give an error, so that’s good progress! But when we tried to use it, no luck. It seems that a function named foo-bar wasn’t defined.

This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer to work in Emacs (like I do), this is a situation where it’s -worth using DrRacket temporarily for its Macro Stepper.

The Macro Stepper says that the use of our macro:

(hyphen-define/wrong1.1 foo bar () #t)

expanded to:

(define (name) #t)

Well that explains it. Instead, we wanted to expand to:

(define (foo-bar) #t)

Our template is using the symbol name but we wanted its +prefer to work mostly in Emacs (like I do), this is a situation where +it’s worth using DrRacket temporarily for its Macro Stepper.

The Macro Stepper says that the use of our macro:

(hyphen-define/wrong1.1 foo bar () #t)

expanded to:

(define (name) #t)

Well that explains it. Instead, we wanted to expand to:

(define (foo-bar) #t)

Our template is using the symbol name but we wanted its value, such as foo-bar in this use of our macro.

A solution here is with-syntax, which lets us say that name is something whose value can be used in our output template:

> (define-syntax (hyphen-define/wrong1.3 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  #'a
                                                                  #'b)))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.3 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Hmm. foo-bar is still not defined. Back to the Macro @@ -190,14 +190,13 @@ Stepper. It says now we’re expanding to:

inside the syntax object, the symbol foo and bar. To get that, we use syntax->datum:

> (define-syntax (hyphen-define/ok1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  (syntax->datum #'a)
                                                                  (syntax->datum #'b))))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok1 foo bar () #t)
> (foo-bar)

#t

And now it works!

By the way, there is a utility function in racket/syntax called format-id that lets us format identifier names more -succinctly. We remember to use for-syntax with -require, since we need it at compile time:

> (require (for-syntax racket/syntax))
> (define-syntax (hyphen-define/ok2 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (format-id stx "~a-~a" #'a #'b)])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok2 bar baz () #t)
> (bar-baz)

#t

Using format-id is convenient as it handles the tedium of -converting from syntax to datum and back again.

Recap: If you want to munge pattern variables for use in the template, -with-syntax is your friend. Just remember you have to use -syntax or #’ on the pattern variables to turn them into -fun size templates, and often also use syntax->datum to get -the interesting value inside. Finally, format-id is -convenient for formatting identifier names.

5 Syntax parameters

"Anaphoric if" or "aif" is a popular macro example. Instead of writing:

(let ([tmp (big-long-calculation)])
  (if tmp
      (foo tmp)
      #f))

You could write:

(aif (big-long-calculation)
     (foo it)
     #f)

In other words, when the condition is true, an it identifier +succinctly. As we’ve learned, we need to require the module +using for-syntax, since we need it at compile time:

> (require (for-syntax racket/syntax))
> (define-syntax (hyphen-define/ok2 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (format-id stx "~a-~a" #'a #'b)])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok2 bar baz () #t)
> (bar-baz)

#t

Using format-id is convenient as it handles the tedium of +converting from syntax to datum and back again.

To review:

  • If you want to munge pattern variables for use in the +template, with-syntax is your friend.

  • You will need to use syntax or #’ on the pattern +variables to turn them into fun size templates.

  • Usually you’ll need to use syntax->datum to get the +interesting value inside.

  • format-id is convenient for formatting identifier +names.

5 Syntax parameters

"Anaphoric if" or "aif" is a popular macro example. Instead of writing:

(let ([tmp (big-long-calculation)])
  (if tmp
      (foo tmp)
      #f))

You could write:

(aif (big-long-calculation)
     (foo it)
     #f)

In other words, when the condition is true, an it identifier is automatically created and set to the value of the condition. This should be easy:

> (define-syntax-rule (aif condition true-expr false-expr)
    (let ([it condition])
      (if it
          true-expr
          false-expr)))
> (aif #t (displayln it) (void))

it: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Wait, what? it is undefined?

It turns out that all along we have been protected from making a certain kind of mistake in our macros. The mistake is if our new @@ -213,29 +212,33 @@ to introduce a magic variable on purpose—such as aif.

The way to do this is with a "syntax parameter", using define-syntax-parameter and syntax-parameterize. You’re probably familiar with regular -parameters in Racket:

> (define current-foo (make-parameter "some default value"))
> (current-foo)

"some default value"

> (parameterize ([current-foo "I have a new value, for now"])
    (current-foo))

"I have a new value, for now"

> (current-foo)

"some default value"

Historically, there are other ways to do this. If you’re -the target audience I’m writing for, you don’t know them yet. I -suggest not bothering to learn them, yet. (Someday if you want to -understand someone else’s older macros, you can learn about them -then.)

That’s a normal parameter. The syntax variation works similarly. The +parameters in Racket:

> (define current-foo (make-parameter "some default value"))
> (current-foo)

"some default value"

> (parameterize ([current-foo "I have a new value, for now"])
    (current-foo))

"I have a new value, for now"

> (current-foo)

"some default value"

That’s a normal parameter. The syntax variation works similarly. The idea is that we’ll define it to mean an error by default. Only inside of our aif will it have a meaningful value:

> (require racket/stxparam)
> (define-syntax-parameter it
    (lambda (stx)
      (raise-syntax-error (syntax-e stx) "can only be used inside aif")))
> (define-syntax-rule (aif condition true-expr false-expr)
    (let ([tmp condition])
      (if tmp
          (syntax-parameterize ([it (make-rename-transformer #'tmp)])
            true-expr)
          false-expr)))
> (aif 10 (displayln it) (void))

10

> (aif #f (displayln it) (void))

If we try to use it outside of an aif form, and it isn’t otherwise defined, we get an error like we want:

> (displayln it)

it: can only be used inside aif

But we can still define it as a normal variable:

> (define it 10)
> it

10

6 Robust macros: syntax-parse

TO-DO. TO-DO. -TO-DO.

7 Other questions

Hopefully I will answer these in the course of the other sections. But -just in case:

7.1 What’s the point of with-syntax?

Done.

7.2 What’s the point of begin-for-syntax?

TO-DO.

7.3 What’s the point of racket/splicing?

TO-DO.

8 References/Acknowledgments

Eli Barzliay wrote a blog post, - Writing -‘syntax-case’ Macros, which explains many key details. However it’s written -especially for people already familiar with "un-hygienic" "defmacro" -style macros. If you’re not familiar with those, it may seem slightly -weird to the extent it’s trying to convince you to change an opinion -you don’t have. Even so, many key details are presented in Eli’s typically -concise, clear fashion.

Eli Barzilay wrote another blog post, +TO-DO.

7 Other questions

Hopefully I will answer these in the course of writing the other +sections. But just in case, here’s a running list:

7.1 What’s the point of with-syntax?

Done.

7.2 What’s the point of begin-for-syntax?

Done.

7.3 What’s the point of racket/splicing?

TO-DO.

8 References and Acknowledgments

Eli Barzliay’s blog post, +Writing +‘syntax-case’ Macros, helped me understand many key details and +concepts. It also inspired me to use a "bottom-up" approach. However +he wrote for a specific audience. If you’re not already familiar with +un-hygienic defmacro style macros, it may seem slightly weird to the +extent it’s trying to convince you to change an opinion you don’t +have. I’m writing for people who don’t have any opinion about macros +at all, except maybe that macros seem scary and daunting.

Eli wrote another blog post, Dirty Looking Hygiene, which explains syntax-parameterize. I relied heavily on that, mostly just updating it since his post was written -before PLT Scheme was renamed to Racket.

9 Epilogue

"Before I had studied Chan (Zen) for thirty years, I saw mountains as +before PLT Scheme was renamed to Racket.

After initially wondering if I was asking the wrong question and +conflating two different issues :), Shriram Krishnamurthi looked at an +early draft and encouraged me to keep going. Sam Tobin-Hochstadt also +encouraged me.

After writing much of this, I noticed that Racket’s documentation had +improved since I last read it. Actually it was the same, and very +good—I’d changed. It’s interesting how much of what we already know +is projected between the lines. That’s what makes it so hard to write +documentation. The only advantage I had was knowing so much less.

9 Epilogue

"Before I had studied Chan (Zen) for thirty years, I saw mountains as mountains, and rivers as rivers. When I arrived at a more intimate knowledge, I came to the point where I saw that mountains are not mountains, and rivers are not rivers. But now that I have got its very