4 Pattern matching: syntax-case and syntax-rules
Most useful syntax transformers work by taking some input syntax, and
rearranging the pieces into something else. As we saw, this is
possible but tedious using list accessors such as
-cadddr. It’s more convenient and less error-prone to use
-match to do pattern-matching.
Historically, syntax-case and
-syntax-rules pattern matching came first. match was
+cadddr. It’s more convenient and less error-prone to use
+match to do pattern-matching.
Historically, syntax-case and
+syntax-rules pattern matching came first. match was
added to Racket later.
It turns out that pattern-matching was one of the first improvements
to be added to the Racket macro system. It’s called
-syntax-case, and has a shorthand for simple situations called
-define-syntax-rule.
Recall our previous example:
Here’s what it looks like using syntax-case:
|
| > (our-if-using-syntax-case #t "true" "false") |
"true" |
Pretty similar, huh? The pattern matching part looks almost exactly
+syntax-case, and has a shorthand for simple situations called
+define-syntax-rule.
Recall our previous example:
Here’s what it looks like using syntax-case:
|
| > (our-if-using-syntax-case #t "true" "false") |
"true" |
Pretty similar, huh? The pattern matching part looks almost exactly
the same. The way we specify the new syntax is simpler. We don’t need
to do quasi-quoting and unquoting. We don’t need to use
-datum->syntax. Instead, we supply a "template", which uses
+datum->syntax. Instead, we supply a "template", which uses
variables from the pattern.
There is a shorthand for simple pattern-matching cases, which expands
-into syntax-case. It’s called define-syntax-rule:
|
| > (our-if-using-syntax-rule #t "true" "false") |
"true" |
Here’s the thing about define-syntax-rule. Because it’s so
-simple, define-syntax-rule is often the first thing people are
+into syntax-case. It’s called define-syntax-rule:
|
| > (our-if-using-syntax-rule #t "true" "false") |
"true" |
Here’s the thing about define-syntax-rule. Because it’s so
+simple, define-syntax-rule is often the first thing people are
taught about macros. But it’s almost deceptively simple. It looks so
much like defining a normal run time function—yet it’s not. It’s
working at compile time, not run time. Worse, the moment you want to
-do more than define-syntax-rule can handle, you can fall off
+do more than define-syntax-rule can handle, you can fall off
a cliff into what feels like complicated and confusing
territory. Hopefully, because we started with a basic syntax
transformer, and worked up from that, we won’t have that problem. We
-can appreciate define-syntax-rule as a convenient shorthand,
+can appreciate define-syntax-rule as a convenient shorthand,
but not be scared of, or confused about, that for which it’s
shorthand.
Most of the materials I found for learning macros, including the
Racket Guide, do a very good job explaining
@@ -30,92 +30,92 @@ Racket Guide, do a very good job explain
patterns and templates work. So I won’t regurgitate that here.
Sometimes, we need to go a step beyond the pattern and template. Let’s
look at some examples, how we can get confused, and how to get it
working.
4.1 Pattern variable vs. template—fight!
Let’s say we want to define a function with a hyphenated name, a-b,
-but we supply the a and b parts separately. The Racket struct
-macro does something like this: (struct foo (field1 field2))
+but we supply the a and b parts separately. The Racket struct
+macro does something like this: (struct foo (field1 field2))
automatically defines a number of functions whose names are variations
on the name foo—such as foo-field1,
foo-field2, foo?, and so on.
So let’s pretend we’re doing something like that. We want to transform
the syntax (hyphen-define a b (args) body) to the syntax
-(define (a-b args) body).
A wrong first attempt is:
|
eval:47:0: a: pattern variable cannot be used outside of a |
template |
in: a |
Huh. We have no idea what this error message means. Well, let’s try to
+(define (a-b args) body).
A wrong first attempt is:
|
eval:47:0: a: pattern variable cannot be used outside of a |
template |
in: a |
Huh. We have no idea what this error message means. Well, let’s try to
work it out. The "template" the error message refers to is the
-#'(define (name args ...) body0 body ...) portion. The
-let isn’t part of that template. It sounds like we can’t use
-a (or b) in the let part.
In fact, syntax-case can have as many templates as you
+#'(define (name args ...) body0 body ...) portion. The
+let isn’t part of that template. It sounds like we can’t use
+a (or b) in the let part.
In fact, syntax-case can have as many templates as you
want. The obvious, required template is the final expression supplying
-the output syntax. But you can use syntax (a.k.a. #’) on a
+the output syntax. But you can use syntax (a.k.a. #’) on a
pattern variable. This makes another template, albeit a small, "fun
-size" template. Let’s try that:
No more error—good! Let’s try to use it:
| > (hyphen-define/wrong1.1 foo bar () #t) |
| > (foo-bar) |
foo-bar: undefined; |
cannot reference an identifier before its definition |
in module: 'program |
Apparently our macro is defining a function with some name other than
+size" template. Let’s try that:
No more error—good! Let’s try to use it:
| > (hyphen-define/wrong1.1 foo bar () #t) |
| > (foo-bar) |
foo-bar: undefined; |
cannot reference an identifier before its definition |
in module: 'program |
Apparently our macro is defining a function with some name other than
foo-bar. Huh.
This is where the Macro Stepper in DrRacket is
invaluable.
Even if you prefer mostly to use Emacs, this
is a situation where it’s definitely worth temporarily using DrRacket
-for its Macro Stepper.
The Macro Stepper says that the use of our macro:
(hyphen-define/wrong1.1 foo bar () #t)
expanded to:
(define (name) #t)
Well that explains it. Instead, we wanted to expand to:
(define (foo-bar) #t)
Our template is using the symbol name but we wanted its
+for its Macro Stepper.
The Macro Stepper says that the use of our macro:
(hyphen-define/wrong1.1 foo bar () #t)
expanded to:
(define (name) #t)
Well that explains it. Instead, we wanted to expand to:
(define (foo-bar) #t)
Our template is using the symbol name but we wanted its
value, such as foo-bar in this use of our macro.
Is there anything we already know that behaves like this—where using
a variable in the template yields its value? Yes: Pattern
variables. Our pattern doesn’t include name because we don’t
expect it in the original syntax—indeed the whole point of this
macro is to create it. So name can’t be in the main
pattern. Fine—let’s make an additional pattern. We can do
-that using an additional, nested syntax-case:
Looks weird? Let’s take a deep breath. Normally our transformer
+that using an additional, nested syntax-case:
Looks weird? Let’s take a deep breath. Normally our transformer
function is given syntax by Racket, and we pass that syntax to
-syntax-case. But we can also create some syntax of our own,
-on the fly, and pass that to syntax-case. That’s all
-we’re doing here. The whole (datum->syntax ...) expression is
+syntax-case. But we can also create some syntax of our own,
+on the fly, and pass that to syntax-case. That’s all
+we’re doing here. The whole (datum->syntax ...) expression is
syntax that we’re creating on the fly. We can give that to
-syntax-case, and match it using a pattern variable named
+syntax-case, and match it using a pattern variable named
name. Voila, we have a new pattern variable. We can use it in
a template, and its value will go in the template.
We might have one more—just one, I promise!—small problem left.
Let’s try to use our new version:
| > (hyphen-define/wrong1.2 foo bar () #t) |
| > (foo-bar) |
foo-bar: undefined; |
cannot reference an identifier before its definition |
in module: 'program |
Hmm. foo-bar is still not defined. Back to the Macro
-Stepper. It says now we’re expanding to:
(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)
Oh right: #'a and #'b are syntax objects. Therefore
(string->symbol (format "~a-~a" #'a #'b))
is the printed form of both syntax objects, joined by a hyphen:
|#<syntax:11:24foo>-#<syntax:11:28 bar>|
Instead we want the datum in the syntax objects, such as the symbols
+Stepper. It says now we’re expanding to:
(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)
Oh right: #'a and #'b are syntax objects. Therefore
(string->symbol (format "~a-~a" #'a #'b))
is the printed form of both syntax objects, joined by a hyphen:
|#<syntax:11:24foo>-#<syntax:11:28 bar>|
Instead we want the datum in the syntax objects, such as the symbols
foo and bar. Which we get using
-syntax->datum:
|
| > (hyphen-define/ok1 foo bar () #t) |
| > (foo-bar) |
#t |
And now it works!
Next, some shortcuts.
Instead of an additional, nested syntax-case, we could use
-with-syntaxAnother name for
-with-syntax could be, "with new pattern variable".. This
-rearranges the syntax-case to look more like a let
+syntax->datum:
|
| > (hyphen-define/ok1 foo bar () #t) |
| > (foo-bar) |
#t |
And now it works!
Next, some shortcuts.
Instead of an additional, nested syntax-case, we could use
+with-syntaxAnother name for
+with-syntax could be, "with new pattern variable".. This
+rearranges the syntax-case to look more like a let
statement—first the name, then the value. Also it’s more convenient
-if we need to define more than one pattern variable.
|
| > (hyphen-define/ok2 foo bar () #t) |
| > (foo-bar) |
#t |
Again, with-syntax is simply syntax-case rearranged:
Whether you use an additional syntax-case or use
-with-syntax, either way you are simply defining additional
+if we need to define more than one pattern variable.
|
| > (hyphen-define/ok2 foo bar () #t) |
| > (foo-bar) |
#t |
Again, with-syntax is simply syntax-case rearranged:
Whether you use an additional syntax-case or use
+with-syntax, either way you are simply defining additional
pattern variables. Don’t let the terminology and structure make it
-seem mysterious.
We know that let doesn’t let us use a binding in a subsequent
-one:
|
a: undefined; |
cannot reference an identifier before its definition |
in module: 'program |
Instead we can nest lets:
| > (let ([a 0]) | | (let ([b a]) | | b)) |
|
0 |
Or use a shorthand for nesting, let*:
Similarly, instead of writing nested with-syntaxs, we can use
-with-syntax*:
One gotcha is that with-syntax* isn’t provided by
-racket/base. We must (require (for-syntax racket/syntax)). Otherwise we may get a rather bewildering error
-message:
...: ellipses not allowed as an expression in: ....
There is a utility function in racket/syntax called
-format-id that lets us format identifier names more
-succinctly than what we did above:
| > (require (for-syntax racket/syntax)) |
|
| > (hyphen-define/ok3 bar baz () #t) |
| > (bar-baz) |
#t |
Using format-id is convenient as it handles the tedium of
+seem mysterious.
We know that let doesn’t let us use a binding in a subsequent
+one:
|
a: undefined; |
cannot reference an identifier before its definition |
in module: 'program |
Instead we can nest lets:
| > (let ([a 0]) | | (let ([b a]) | | b)) |
|
0 |
Or use a shorthand for nesting, let*:
Similarly, instead of writing nested with-syntaxs, we can use
+with-syntax*:
One gotcha is that with-syntax* isn’t provided by
+racket/base. We must (require (for-syntax racket/syntax)). Otherwise we may get a rather bewildering error
+message:
...: ellipses not allowed as an expression in: ....
There is a utility function in racket/syntax called
+format-id that lets us format identifier names more
+succinctly than what we did above:
| > (require (for-syntax racket/syntax)) |
|
| > (hyphen-define/ok3 bar baz () #t) |
| > (bar-baz) |
#t |
Using format-id is convenient as it handles the tedium of
converting from syntax to symbol datum to string ... and all the way
back.
4.1.4 Another example
Finally, here’s a variation that accepts an arbitary number of name
-parts to be joined with hyphens:
| > (require (for-syntax racket/string racket/syntax)) |
|
| > (hyphen-define* (foo bar baz) (v) (* 2 v)) |
| > (foo-bar-baz 50) |
100 |
To review:
You can’t use a pattern variable outside of a template. But
-you can use syntax or #’ on a pattern variable to make
+parts to be joined with hyphens:
| > (require (for-syntax racket/string racket/syntax)) |
|
| > (hyphen-define* (foo bar baz) (v) (* 2 v)) |
| > (foo-bar-baz 50) |
100 |
To review:
You can’t use a pattern variable outside of a template. But
+you can use syntax or #’ on a pattern variable to make
an ad hoc, "fun size" template.
If you want to munge pattern variables for use in the
-template, with-syntax is your friend, because it lets you
-create new pattern variables.
Usually you’ll need to use syntax->datum to get the
-interesting value inside.
format-id is convenient for formatting identifier
-names.
4.2 Making our own struct
Let’s apply what we just learned to a more-realistic example. We’ll
-pretend that Racket doesn’t already have a struct
+template, with-syntax is your friend, because it lets you
+create new pattern variables.
Usually you’ll need to use syntax->datum to get the
+interesting value inside.
format-id is convenient for formatting identifier
+names.
4.2 Making our own struct
Let’s apply what we just learned to a more-realistic example. We’ll
+pretend that Racket doesn’t already have a struct
capability. Fortunately, we can write a macro to provide our own
system for defining and using structures. To keep things simple, our
structure will be immutable (read-only) and it won’t support
-inheritance.
Given a structure declaration like:
(our-struct name (field1 field2 ...))
We need to define some procedures:
A constructor procedure whose name is the struct name. We’ll
-represent structures as a vector. The structure name will be
+inheritance.
Given a structure declaration like:
(our-struct name (field1 field2 ...))
We need to define some procedures:
A constructor procedure whose name is the struct name. We’ll
+represent structures as a vector. The structure name will be
element zero. The fields will be elements one onward.
A predicate, whose name is the struct name with ?
appended.
For each field, an accessor procedure to get its value. These
will be named struct-field (the name of the struct, a hyphen, and the
-field name).
| > (require (for-syntax racket/syntax)) |
|
| ; Test it out |
| > (require rackunit) |
| > (our-struct foo (a b)) |
| > (define s (foo 1 2)) |
| > (check-true (foo? s)) |
| > (check-false (foo? 1)) |
| > (check-equal? (foo-a s) 1) |
| > (check-equal? (foo-b s) 2) |
|
| ; The tests passed. |
| ; Next, what if someone tries to declare: |
| > (our-struct "blah" ("blah" "blah")) |
format-id: contract violation |
expected: (or/c string? symbol? identifier? keyword? char? |
number?) |
given: #<syntax:83:0 "blah"> |
The error message is not very helpful. It’s coming from
-format-id, which is a private implementation detail of our macro.
You may know that a syntax-case clause can take an
-optional "guard" or "fender" expression. Instead of
[pattern template]
It can be:
[pattern guard template]
Let’s add a guard expression to our clause:
| > (require (for-syntax racket/syntax)) |
|
| ; Now the same misuse gives a better error message: |
| > (our-struct "blah" ("blah" "blah")) |
eval:86:0: our-struct: not an identifier |
at: "blah" |
in: (our-struct "blah" ("blah" "blah")) |
Later, we’ll see how syntax-parse makes it even easier to
+field name).
| > (require (for-syntax racket/syntax)) |
|
| ; Test it out |
| > (require rackunit) |
| > (our-struct foo (a b)) |
| > (define s (foo 1 2)) |
| > (check-true (foo? s)) |
| > (check-false (foo? 1)) |
| > (check-equal? (foo-a s) 1) |
| > (check-equal? (foo-b s) 2) |
|
| ; The tests passed. |
| ; Next, what if someone tries to declare: |
| > (our-struct "blah" ("blah" "blah")) |
format-id: contract violation |
expected: (or/c string? symbol? identifier? keyword? char? |
number?) |
given: #<syntax:83:0 "blah"> |
The error message is not very helpful. It’s coming from
+format-id, which is a private implementation detail of our macro.
You may know that a syntax-case clause can take an
+optional "guard" or "fender" expression. Instead of
[pattern template]
It can be:
[pattern guard template]
Let’s add a guard expression to our clause:
| > (require (for-syntax racket/syntax)) |
|
| ; Now the same misuse gives a better error message: |
| > (our-struct "blah" ("blah" "blah")) |
eval:86:0: our-struct: not an identifier |
at: "blah" |
in: (our-struct "blah" ("blah" "blah")) |
Later, we’ll see how syntax-parse makes it even easier to
check usage and provide helpful messages about mistakes.
4.3 Using dot notation for nested hash lookups
The previous two examples used a macro to define functions whose names
were made by joining identifiers provided to the macro. This example
does the opposite: The identifier given to the macro is split into
pieces.
If you write programs for web services you deal with JSON, which is
represented in Racket by a jsexpr?. JSON often has
dictionaries that contain other dictionaries. In a jsexpr?
-these are represented by nested hasheq tables:
In JavaScript you can use dot notation:
In Racket it’s not so convenient:
(hash-ref (hash-ref (hash-ref js 'a) 'b) 'c)
We can write a helper function to make this a bit cleaner:
| ; This helper function: |
|
| ; Lets us say: |
| > (hash-refs js '(a b c)) |
"value" |
That’s better. Can we go even further and use a dot notation somewhat
-like JavaScript?
| ; This macro: |
| > (require (for-syntax racket/syntax)) |
|
| ; Gives us "sugar" to say this: |
| > (hash.refs js.a.b.c) |
"value" |
| ; Try finding a key that doesn't exist: |
| > (hash.refs js.blah) |
#f |
| ; Try finding a key that doesn't exist, specifying the default: |
| > (hash.refs js.blah 'did-not-exist) |
'did-not-exist |
It works!
We’ve started to appreciate that our macros should give helpful
-messages when used in error. Let’s try to do that here.
| > (require (for-syntax racket/syntax)) |
|
| ; See if we catch each of the misuses |
| > (hash.refs) |
eval:96:0: hash.refs: Expected (hash.key0[.key1 ...] |
[default]) |
at: chain |
in: (hash.refs) |
| > (hash.refs 0) |
eval:98:0: hash.refs: Expected (hash.key0[.key1 ...] |
[default]) |
at: 0 |
in: (hash.refs 0 #f) |
| > (hash.refs js) |
eval:99:0: hash.refs: Expected hash.key |
at: js |
in: (hash.refs js #f) |
| > (hash.refs js.) |
eval:100:0: hash.refs: Expected hash.key |
at: js. |
in: (hash.refs js. #f) |
Not too bad. Of course, the version with error-checking is quite a bit
+these are represented by nested hasheq tables:
In JavaScript you can use dot notation:
In Racket it’s not so convenient:
(hash-ref (hash-ref (hash-ref js 'a) 'b) 'c)
We can write a helper function to make this a bit cleaner:
| ; This helper function: |
|
| ; Lets us say: |
| > (hash-refs js '(a b c)) |
"value" |
That’s better. Can we go even further and use a dot notation somewhat
+like JavaScript?
| ; This macro: |
| > (require (for-syntax racket/syntax)) |
|
| ; Gives us "sugar" to say this: |
| > (hash.refs js.a.b.c) |
"value" |
| ; Try finding a key that doesn't exist: |
| > (hash.refs js.blah) |
#f |
| ; Try finding a key that doesn't exist, specifying the default: |
| > (hash.refs js.blah 'did-not-exist) |
'did-not-exist |
It works!
We’ve started to appreciate that our macros should give helpful
+messages when used in error. Let’s try to do that here.
| > (require (for-syntax racket/syntax)) |
|
| ; See if we catch each of the misuses |
| > (hash.refs) |
eval:96:0: hash.refs: Expected (hash.key0[.key1 ...] |
[default]) |
at: chain |
in: (hash.refs) |
| > (hash.refs 0) |
eval:98:0: hash.refs: Expected (hash.key0[.key1 ...] |
[default]) |
at: 0 |
in: (hash.refs 0 #f) |
| > (hash.refs js) |
eval:99:0: hash.refs: Expected hash.key |
at: js |
in: (hash.refs js #f) |
| > (hash.refs js.) |
eval:100:0: hash.refs: Expected hash.key |
at: js. |
in: (hash.refs js. #f) |
Not too bad. Of course, the version with error-checking is quite a bit
longer. Error-checking code generally tends to obscure the logic, and
-does here. Fortuantely we’ll soon see how syntax-parse can
+does here. Fortuantely we’ll soon see how syntax-parse can
help mitigate that, in much the same way as contracts in normal
Racket or types in Typed Racket.
Maybe we’re not convinced that writing (hash.refs js.a.b.c)
is really clearer than (hash-refs js '(a b c)). Maybe we
won’t actually use this approach. But the Racket macro system makes it
-a possible choice.
-