Fear of Macros
1 Preface
I learned Racket after 25 years of mostly using C and C++.
Some psychic whiplash resulted.
"All the parentheses" was actually not a big deal. Instead, the first +
1 Preface
I learned Racket after 25 years of mostly using C and C++.
Some psychic whiplash resulted.
"All the parentheses" was actually not a big deal. Instead, the first mind warp was functional programming. Before long I wrapped my brain around it, and went on to become comfortable and effective with many other aspects and features of Racket.
But two final frontiers remained: Macros and continuations.
I found that simple macros were easy and understandable, plus there @@ -47,12 +47,12 @@ similar for macro. There is. One of the more-recent Racket macro enhancements is syntax-parse.
3 Transformers
YOU ARE INSIDE A ROOM. |
THERE ARE KEYS ON THE GROUND. |
THERE IS A SHINY BRASS LAMP NEARBY. |
|
IF YOU GO THE WRONG WAY, YOU WILL BECOME |
HOPELESSLY LOST AND CONFUSED. |
|
> pick up the keys |
|
YOU HAVE A SYNTAX TRANSFORMER |
3.1 What is a syntax transformer?
A syntax transformer is not one of the トランスフォーマ transformers.
Instead, it is simply a function. The function takes syntax and returns syntax. It transforms syntax.
Here’s a transformer function that ignores its input syntax, and -always outputs syntax for a string literal:
> (define-syntax foo (lambda (stx) #'"I am foo")) > (foo) "I am foo"
When we use define-syntax, we’re making a transformer +always outputs syntax for a string literal:
> (define-syntax foo (lambda (stx) #'"I am foo"))
Using it:
> (foo) "I am foo"
When we use define-syntax, we’re making a transformer binding. This tells the Racket compiler, "Whenever you encounter a chunk of syntax starting with foo, please give it to my transformer function, and replace it with the syntax I give back -to you." So Racket will give anything that looks like (foo ...) to our function, and we can change it. Much like a -search-and-replace.
Maybe you know that the usual way to define a function in Racket:
(define (f x) ...)
is shorthand for:
(define f (lambda (x) ...))
That shorthand lets you avoid typing lambda and some parentheses.
Well there is a similar shorthand for define-syntax:
> (define-syntax (also-foo stx) #'"I am also foo") > (also-foo) "I am also foo"
What we want to remember is that this is simply shorthand. We are +to you." So Racket will give anything that looks like (foo ...) to our function, and we can return new syntax to use +instead. Much like a search-and-replace.
Maybe you know that the usual way to define a function in Racket:
(define (f x) ...)
is shorthand for:
(define f (lambda (x) ...))
That shorthand lets you avoid typing lambda and some parentheses.
Well there is a similar shorthand for define-syntax:
> (define-syntax (also-foo stx) #'"I am also foo") > (also-foo) "I am also foo"
What we want to remember is that this is simply shorthand. We are still defining a transformer function, which takes syntax and returns syntax. Everything we do with macros, will be built on top of this basic idea. It’s not magic.
Speaking of shorthand, there is also a shorthand for syntax, @@ -61,14 +61,25 @@ string literal. How about returning ((say-hi), and sees it has a transformer function for that. It calls our function with the old syntax, and we return the new syntax, which is used to evaluate and run our program.
3.2 What’s the input?
Our examples so far ignored the input syntax, and output a fixed -syntax. But usually we want to transform the input to something else.
Let’s start by looking closely at what the input actually is:
> (define-syntax (show-me stx) (print stx) #'(void)) > (show-me '(i am a list)) #<syntax:10:0 (show-me (quote (i am a list)))>
The (print stx) shows what our transformer is given: a syntax +syntax. But instead of throwing away the input, usually we want to +transform the input.
Let’s start by looking closely at what the input actually is:
> (define-syntax (show-me stx) (print stx) #'(void)) > (show-me '(i am a list)) #<syntax:10:0 (show-me (quote (i am a list)))>
The (print stx) shows what our transformer is given: a syntax object.
A syntax object consists of several things. The first part is the
s-expression representing the code, such as '(i am a list). Racket (and Scheme and Lisp) expressions are s-expressions—
Racket syntax is also decorated with some interesting information such as the source file, line number, and column. Finally, it has information about lexical scoping (which you don’t need to worry about -now, but will turn out to be important later.)
There are a variety of functions available to access a syntax object:
> (define stx #'(if x (list "true") #f)) > (syntax->datum stx) '(if x (list "true") #f)
> (syntax-e stx) '(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)
> (syntax->list stx) '(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)
> (syntax-source stx) 'eval
> (syntax-line stx) 11
> (syntax-column stx) 0
When we want to transform syntax, we’ll generally take the pieces we +now, but will turn out to be important later.)
There are a variety of functions available to access a syntax object. +Let’s define a piece of syntax:
> (define stx #'(if x (list "true") #f)) > stx #<syntax:11:0 (if x (list "true") #f)>
Now let’s use functions that access the syntax object. The source +information functions:
(syntax-source stx) is returning 'eval, +only becaue of how I’m generating this documentation, using an +evaluator to run code snippets in Scribble. Normally this would be +somthing like "my-file.rkt".
> (syntax-source stx) 'eval
> (syntax-line stx) 11
> (syntax-column stx) 0
More interesting is the syntax "stuff" itself. syntax->datum +converts it completely into an s-expression:
> (syntax->datum stx) '(if x (list "true") #f)
Whereas syntax-e only goes "one level down". It may return a +list that has syntax objects:
> (syntax-e stx) '(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)
Each of those syntax objects could be converted by syntax-e,
+and so on recursively—
In most cases, syntax->list gives the same result as +syntax-e:
> (syntax->list stx) '(#<syntax:11:0 if> #<syntax:11:0 x> #<syntax:11:0 (list "true")> #<syntax:11:0 #f>)
When would syntax-e and syntax->list differ? Let’s +not get side-tracked now.
When we want to transform syntax, we’ll generally take the pieces we were given, maybe rearrange their order, perhaps change some of the pieces, and often introduce brand-new pieces.
3.3 Actually transforming the input
Let’s write a transformer function that reverses the syntax it was given:
> (define-syntax (reverse-me stx) (datum->syntax stx (reverse (cdr (syntax->datum stx))))) > (reverse-me "backwards" "am" "i" values) "i"
"am"
"backwards"
Understand Yoda, can we. Great, but how does this work?
First we take the input syntax, and give it to @@ -79,9 +90,9 @@ list:
reverse changes it to (values "i" "am" "backwards"):
> (reverse (cdr '("backwards" "am" "i" values))) '(values "i" "am")
Finally we use syntax->datum to convert this back to syntax:
> (datum->syntax #f '(values "i" "am" "backwards")) #<syntax (values "i" "am" "backwards")>
That’s what our transformer function gives back to the Racket compiler, and that syntax is evaluated:
> (values "i" "am" "backwards") "i"
"am"
"backwards"
3.4 Compile time vs. run time
(define-syntax (foo stx) (make-pipe) ;This is not run time. #'(void)) Normal Racket code runs at ... run time. Duh.
Instead of "compile time vs. run time", you may hear it -described as "syntax phase vs. runtime phase". Same difference.
But a syntax transformer is run by the Racket compiler, as part of the -process of parsing, expanding and understanding your code. In other -words, your syntax transformer function is evaluated at compile time.
This aspect of macros lets you do things that simply aren’t possible +described as "syntax phase vs. runtime phase". Same difference.
But a syntax transformer is called by Racket as part of the process of +parsing, expanding, and compiling our program. In other words, our +syntax transformer function is evaluated at compile time.
This aspect of macros lets you do things that simply aren’t possible in normal code. One of the classic examples is something like the Racket form, if:
(if <condition> <true-expression> <false-expression>)
If we implemented if as a function, all of the arguments would be evaluated before being provided to the function.
> (define (our-if condition true-expr false-expr) (cond [condition true-expr] [else false-expr]))
> (our-if #t "true" "false") "true"
That seems to work. However, how about this:
> (define (display-and-return x) (displayln x) x)
> (our-if #t (display-and-return "true") (display-and-return "false"))
true
false
"true"
One answer is that functional programming is good, and @@ -95,7 +106,7 @@ transformer can rearrange the syntax – rewrite the code – at compile time. The pieces of syntax are moved around, but they aren’t actually evaluated until run time.
Here is one way to do this:
> (define-syntax (our-if-v2 stx) (define xs (syntax->list stx)) (datum->syntax stx `(cond [,(cadr xs) ,(caddr xs)] [else ,(cadddr xs)])))
> (our-if-v2 #t (display-and-return "true") (display-and-return "false")) true
"true"
> (our-if-v2 #f (display-and-return "true") (display-and-return "false")) false
"false"
That gave the right answer. But how? Let’s pull out the transformer function itself, and see what it did. We start with an example of some -input syntax:
> (define stx #'(our-if-v2 #t "true" "false")) > (displayln stx) #<syntax:31:0 (our-if-v2 #t "true" "false")>
1. We take the original syntax, and use syntax->datum to +input syntax:
> (define stx #'(our-if-v2 #t "true" "false")) > (displayln stx) #<syntax:32:0 (our-if-v2 #t "true" "false")>
1. We take the original syntax, and use syntax->datum to change it into a plain Racket list:
> (define xs (syntax->datum stx)) > (displayln xs) (our-if-v2 #t true false)
2. To change this into a Racket cond form, we need to take the three interesting pieces—
the condition, true-expression, and false-expression— from the list using cadr, caddr, @@ -169,7 +180,7 @@ automatically defines a number of functions whose names are variations on the name foo— such as foo-field1, foo-field2, foo?, and so on. So let’s pretend we’re doing something like that. We want to transform the syntax (hyphen-define a b (args) body) to the syntax -(define (a-b args) body).
A wrong first attempt is:
> (define-syntax (hyphen-define/wrong1 stx) (syntax-case stx () [(_ a b (args ...) body0 body ...) (let ([name (string->symbol (format "~a-~a" a b))]) #'(define (name args ...) body0 body ...))])) eval:46:0: a: pattern variable cannot be used outside of a
template
in: a
Huh. We have no idea what this error message means. Well, let’s try to +(define (a-b args) body).
A wrong first attempt is:
> (define-syntax (hyphen-define/wrong1 stx) (syntax-case stx () [(_ a b (args ...) body0 body ...) (let ([name (string->symbol (format "~a-~a" a b))]) #'(define (name args ...) body0 body ...))])) eval:47:0: a: pattern variable cannot be used outside of a
template
in: a
Huh. We have no idea what this error message means. Well, let’s try to work it out. The "template" the error message refers to is the #'(define (name args ...) body0 body ...) portion. The let isn’t part of that template. It sounds like we can’t use @@ -194,8 +205,8 @@ succinctly. As we’ve learned, we need to for-syntax, since we need it at compile time:
> (require (for-syntax racket/syntax))
> (define-syntax (hyphen-define/ok2 stx) (syntax-case stx () [(_ a b (args ...) body0 body ...) (with-syntax ([name (format-id stx "~a-~a" #'a #'b)]) #'(define (name args ...) body0 body ...))])) > (hyphen-define/ok2 bar baz () #t) > (bar-baz) #t
Using format-id is convenient as it handles the tedium of converting from syntax to datum and back again.
To review:
If you want to munge pattern variables for use in the template, with-syntax is your friend.
You will need to use syntax or #’ on the pattern -variables to turn them into fun size templates.
Usually you’ll need to use syntax->datum to get the -interesting value inside.
format-id is convenient for formatting identifier +variables to turn them into "fun size" templates.
Usually you’ll also need to use syntax->datum to get +the interesting value inside.
format-id is convenient for formatting identifier names.
5 Syntax parameters
"Anaphoric if" or "aif" is a popular macro example. Instead of writing:
(let ([tmp (big-long-calculation)]) (if tmp (foo tmp) #f)) You could write:
(aif (big-long-calculation) (foo it) #f) In other words, when the condition is true, an it identifier is automatically created and set to the value of the condition. This should be easy:
> (define-syntax-rule (aif condition true-expr false-expr) (let ([it condition]) (if it true-expr false-expr))) > (aif #t (displayln it) (void)) it: undefined;
cannot reference an identifier before its definition
in module: 'program
Wait, what? it is undefined?
It turns out that all along we have been protected from making a diff --git a/main.rkt b/main.rkt index d1529d2..3091909 100644 --- a/main.rkt +++ b/main.rkt @@ -140,6 +140,11 @@ always outputs syntax for a string literal: (define-syntax foo (lambda (stx) (syntax "I am foo"))) +] + +Using it: + +@i[ (foo) ] @@ -148,8 +153,8 @@ When we use @racket[define-syntax], we're making a transformer encounter a chunk of syntax starting with @racket[foo], please give it to my transformer function, and replace it with the syntax I give back to you." So Racket will give anything that looks like @racket[(foo -...)] to our function, and we can change it. Much like a -search-and-replace. +...)] to our function, and we can return new syntax to use +instead. Much like a search-and-replace. Maybe you know that the usual way to define a function in Racket: @@ -201,7 +206,8 @@ which is used to evaluate and run our program. @subsection{What's the input?} Our examples so far ignored the input syntax, and output a fixed -syntax. But usually we want to transform the input to something else. +syntax. But instead of throwing away the input, usually we want to +transform the input. Let's start by looking closely at what the input actually @italic{is}: @@ -226,18 +232,56 @@ as the source file, line number, and column. Finally, it has information about lexical scoping (which you don't need to worry about now, but will turn out to be important later.) -There are a variety of functions available to access a syntax object: +There are a variety of functions available to access a syntax object. +Let's define a piece of syntax: @i[ (define stx #'(if x (list "true") #f)) -(syntax->datum stx) -(syntax-e stx) -(syntax->list stx) +stx +] + +Now let's use functions that access the syntax object. The source +information functions: + +@margin-note{@racket[(syntax-source stx)] is returning @racket['eval], +only becaue of how I'm generating this documentation, using an +evaluator to run code snippets in Scribble. Normally this would be +somthing like "my-file.rkt".} + +@i[ (syntax-source stx) (syntax-line stx) (syntax-column stx) ] +More interesting is the syntax "stuff" itself. @racket[syntax->datum] +converts it completely into an s-expression: + +@i[ +(syntax->datum stx) +] + +Whereas @racket[syntax-e] only goes "one level down". It may return a +list that has syntax objects: + +@i[ +(syntax-e stx) +] + +Each of those syntax objects could be converted by @racket[syntax-e], +and so on recursively---which is what @racket[syntax->datum] does. + +In most cases, @racket[syntax->list] gives the same result as +@racket[syntax-e]: + +@i[ +(syntax->list stx) +] + +When would @racket[syntax-e] and @racket[syntax->list] differ? Let's +not get side-tracked now. + + When we want to transform syntax, we'll generally take the pieces we were given, maybe rearrange their order, perhaps change some of the pieces, and often introduce brand-new pieces. @@ -300,9 +344,9 @@ Normal Racket code runs at ... run time. Duh. @margin-note{Instead of "compile time vs. run time", you may hear it described as "syntax phase vs. runtime phase". Same difference.} -But a syntax transformer is run by the Racket compiler, as part of the -process of parsing, expanding and understanding your code. In other -words, your syntax transformer function is evaluated at compile time. +But a syntax transformer is called by Racket as part of the process of +parsing, expanding, and compiling our program. In other words, our +syntax transformer function is evaluated at compile time. This aspect of macros lets you do things that simply aren't possible in normal code. One of the classic examples is something like the