From ab6ca8cd396824d8320c3e402953d4e2276dd6e3 Mon Sep 17 00:00:00 2001 From: Greg Hendershott Date: Thu, 1 Nov 2012 16:52:51 -0400 Subject: [PATCH] Regen html --- index.html | 79 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 52 insertions(+), 27 deletions(-) diff --git a/index.html b/index.html index 15cce7c..cee2ea6 100644 --- a/index.html +++ b/index.html @@ -1,6 +1,6 @@ Fear of Macros
►Fear of Macros
1 Preface
2 The plan of attack
3 Transformers
3.1 What is a syntax transformer?
3.2 What’s the input?
3.3 Actually transforming the input
3.4 Compile time vs. run time
3.5 begin-for-syntax
4 Pattern matching: syntax-case and syntax-rules
4.1 "A pattern variable can’t be used outside of a template"
4.2 Making our own struct
4.3 Using dot notation for nested hash lookups
5 Syntax parameters
6 Robust macros: syntax-parse
7 What’s the point of splicing-let?
8 References and Acknowledgments
9 Epilogue

Fear of Macros

Greg Hendershott

-
Copyright (c) 2012 by Greg Hendershott. All rights reserved.
Last updated 2012-11-01 12:11:35
Feedback and corrections are welcome here.

Contents:

    1 Preface

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What’s the input?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

      3.5 begin-for-syntax

    4 Pattern matching: syntax-case and syntax-rules

      4.1 "A pattern variable can’t be used outside of a template"

      4.2 Making our own struct

      4.3 Using dot notation for nested hash lookups

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 What’s the point of splicing-let?

    8 References and Acknowledgments

    9 Epilogue

1 Preface

I learned Racket after 25 years of mostly using C and C++.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first +

Copyright (c) 2012 by Greg Hendershott. All rights reserved.
Last updated 2012-11-01 16:52:44
Feedback and corrections are welcome here.

Contents:

    1 Preface

    2 The plan of attack

    3 Transformers

      3.1 What is a syntax transformer?

      3.2 What’s the input?

      3.3 Actually transforming the input

      3.4 Compile time vs. run time

      3.5 begin-for-syntax

    4 Pattern matching: syntax-case and syntax-rules

      4.1 "A pattern variable can’t be used outside of a template"

      4.2 Making our own struct

      4.3 Using dot notation for nested hash lookups

    5 Syntax parameters

    6 Robust macros: syntax-parse

    7 What’s the point of splicing-let?

    8 References and Acknowledgments

    9 Epilogue

1 Preface

I learned Racket after 25 years of mostly using C and C++.

Some psychic whiplash resulted.

"All the parentheses" was actually not a big deal. Instead, the first mind warp was functional programming. Before long I wrapped my brain around it, and went on to become comfortable and effective with many other aspects and features of Racket.

But two final frontiers remained: Macros and continuations.

I found that simple macros were easy and understandable, plus there @@ -188,37 +188,62 @@ work it out. The "template" the error message refers to is the want. The obvious, required template is the final expression supplying the output syntax. But you can use syntax (a.k.a. #’) on a pattern variable. This makes another template, albeit a small, "fun -size" template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))

No more error—good! Let’s try to use it:

> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

It seems we’re defining a function with a name other than -foo-bar?

This is where the Macro Stepper in DrRacket is invaluable. Even if you -prefer mostly to use Emacs, this is a situation where it’s worth using -DrRacket at least temporarily for its Macro Stepper.

The Macro Stepper says that the use of our macro:

(hyphen-define/wrong1.1 foo bar () #t)

expanded to:

(define (name) #t)

Well that explains it. Instead, we wanted to expand to:

(define (foo-bar) #t)

Our template is using the symbol name but we wanted its -value, such as foo-bar in this use of our macro.

A solution here is with-syntaxYou could -consider with-syntax to mean, "define pattern variables"., -which lets us say that name is something whose value can be -used in our output template. In effect, it lets us say that -name is an additional pattern variable.

> (define-syntax (hyphen-define/wrong1.3 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  #'a
                                                                  #'b)))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/wrong1.3 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Hmm. foo-bar is still not defined. Back to the Macro -Stepper. It says now we’re expanding to:

(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)

Oh right: #'a and #'b are syntax objects, and -format is printing them as such. Instead we want the datum in -the syntax objects (the symbols foo and bar). Let’s -use syntax->datum:

> (define-syntax (hyphen-define/ok1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  (syntax->datum #'a)
                                                                  (syntax->datum #'b))))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok1 foo bar () #t)
> (foo-bar)

#t

And now it works!

By the way, there is a utility function in racket/syntax -called format-id that lets us format identifier names more +size" template. Let’s try that:

> (define-syntax (hyphen-define/wrong1.1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (let ([name (string->symbol (format "~a-~a" #'a #'b))])
         #'(define (name args ...)
             body0 body ...))]))

No more error—good! Let’s try to use it:

> (hyphen-define/wrong1.1 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Apparently our macro is defining a function with some name other than +foo-bar. Huh.

This is where the Macro Stepper in DrRacket is +invaluable.

Even if you prefer mostly to use Emacs, this +is a situation where it’s definitely worth temporarily using DrRacket +for its Macro Stepper.

The Macro Stepper says that the use of our macro:

(hyphen-define/wrong1.1 foo bar () #t)

expanded to:

(define (name) #t)

Well that explains it. Instead, we wanted to expand to:

(define (foo-bar) #t)

Our template is using the symbol name but we wanted its +value, such as foo-bar in this use of our macro.

Can we think of something we already know that behaves like +this—where using a variable in the template yields its value? Sure +we do: Pattern variables. Our pattern doesn’t include name +because we don’t expect it in the original syntax—indeed the whole +point of this macro is to create it. So name can’t be in the +main pattern. Fine—let’s make an additional pattern. We can +do that using an additional, nested syntax-case:

> (define-syntax (hyphen-define/wrong1.2 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (syntax-case (datum->syntax stx
                                   (string->symbol (format "~a-~a" #'a #'b))) ()
         [name #'(define (name args ...)
                   body0 body ...)])]))

Looks weird? Let’s take a deep breath. Normally our transformer +function is given syntax by Racket, and we pass that syntax to +syntax-case. But we can also create some syntax of our own, +on the fly, and pass that to syntax-case. That’s all +we’re doing here. The whole (datum->syntax ...) expression is +syntax that we’re creating on the fly. We can give that to +syntax-case, and match it using a pattern variable named +name. Voila, we have a new pattern variable. We can use it in +a template, and its value will go in the template.

We might have one more—just one, I promise!—small problem left. +Let’s try to use our new version:

> (hyphen-define/wrong1.2 foo bar () #t)
> (foo-bar)

foo-bar: undefined;

 cannot reference an identifier before its definition

  in module: 'program

Hmm. foo-bar is still not defined. Back to the Macro +Stepper. It says now we’re expanding to:

(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)

Oh right: #'a and #'b are syntax objects. Therefore

(string->symbol (format "~a-~a" #'a #'b))

is something like

|#<syntax:11:24foo>-#<syntax:11:28 bar>|

—the printed form of both syntax objects, joined by a hyphen.

Instead we want the datum in the syntax objects (such as the symbols +foo and bar). Let’s use syntax->datum to +get it:

> (define-syntax (hyphen-define/ok1 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (syntax-case (datum->syntax stx
                                   (string->symbol (format "~a-~a"
                                                           (syntax->datum #'a)
                                                           (syntax->datum #'b)))) ()
         [name #'(define (name args ...)
                   body0 body ...)])]))
> (hyphen-define/ok1 foo bar () #t)
> (foo-bar)

#t

And now it works!

Now for two shortcuts.

Instead of an additional, nested syntax-case we could use +with-syntaxAnother name for +with-syntax could be, "define pattern variable".. This +rearranges the syntax-case to look more like a let +statement—first the name, then the value. Also it’s more convenient +if we need to define more than one pattern variable.

> (define-syntax (hyphen-define/ok2 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (datum->syntax stx
                                          (string->symbol (format "~a-~a"
                                                                  (syntax->datum #'a)
                                                                  (syntax->datum #'b))))])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok2 foo bar () #t)
> (foo-bar)

#t

Whether you use an additional syntax-case or use +with-syntax, either way you are simply defining an additional +pattern variable. Don’t let the terminology and structure make it seem +mysterious.

Also, there is a utility function in racket/syntax called +format-id that lets us format identifier names more succinctly. As we’ve learned, we need to require the module -using for-syntax, since we need it at compile time:

> (require (for-syntax racket/syntax))
> (define-syntax (hyphen-define/ok2 stx)
    (syntax-case stx ()
      [(_ a b (args ...) body0 body ...)
       (with-syntax ([name (format-id stx "~a-~a" #'a #'b)])
         #'(define (name args ...)
             body0 body ...))]))
> (hyphen-define/ok2 bar baz () #t)
> (bar-baz)

#t

Using format-id is convenient as it handles the tedium of -converting from syntax to datum and back again.

To review:

4.2 Making our own struct

In this example we’ll pretend that Racket doesn’t already have a -struct capability. Fortunately, we can define a macro to -provide this feature. To keep things simple, our structure will be -immutable (read-only) and it won’t support inheritance.

Given a structure declaration like:

(our-struct name (field1 field2 ...))

We need to define some procedures.

> (require (for-syntax racket/syntax))
> (define-syntax (our-struct stx)
    (syntax-case stx ()
      [(_ id (fields ...))
       (with-syntax ([pred-id (format-id stx "~a?" #'id)])
         #`(begin
             ; Define a constructor.
             (define (id fields ...)
               (apply vector (cons 'id  (list fields ...))))
             ; Define a predicate.
             (define (pred-id v)
               (and (vector? v)
                    (eq? (vector-ref v 0) 'id)))
             ; Define an accessor for each field.
             #,@(for/list ([x (syntax->list #'(fields ...))]
                           [n (in-naturals 1)])
                  (with-syntax ([acc-id (format-id stx "~a-~a" #'id x)]
                                [ix n])
                    #`(define (acc-id v)
                        (unless (pred-id v)
                          (error 'acc-id "~a is not a ~a struct" v 'id))
                        (vector-ref v ix))))))]))
; Test it out
> (require rackunit)
> (our-struct foo (a b))
> (define s (foo 1 2))
> (check-true (foo? s))
> (check-false (foo? 1))
> (check-equal? (foo-a s) 1)
> (check-equal? (foo-b s) 2)
> (check-exn exn:fail?
             (lambda () (foo-a "furble")))
; The tests passed.
; Next, what if someone tries to declare:
> (our-struct "blah" ("blah" "blah"))

format-id: contract violation

  expected: (or/c string? symbol? identifier? keyword? char?

number?)

  given: #<syntax:78:0 "blah">

The error message is not very helpful. It’s coming from format-id, which is a private implementation detail of our macro.

You may know that a syntax-case clause can take an -optional "guard" or "fender" expression. Instead of

[pattern template]

It can be:

[pattern guard template]

Let’s add a guard expression to our clause:

> (require (for-syntax racket/syntax))
> (define-syntax (our-struct stx)
    (syntax-case stx ()
      [(_ id (fields ...))
       ; Guard or "fender" expression:
       (for-each (lambda (x)
                   (unless (identifier? x)
                     (raise-syntax-error #f "not an identifier" stx x)))
                 (cons #'id (syntax->list #'(fields ...))))
       (with-syntax ([pred-id (format-id stx "~a?" #'id)])
         #`(begin
             ; Define a constructor.
             (define (id fields ...)
               (apply vector (cons 'id  (list fields ...))))
             ; Define a predicate.
             (define (pred-id v)
               (and (vector? v)
                    (eq? (vector-ref v 0) 'id)))
             ; Define an accessor for each field.
             #,@(for/list ([x (syntax->list #'(fields ...))]
                           [n (in-naturals 1)])
                  (with-syntax ([acc-id (format-id stx "~a-~a" #'id x)]
                                [ix n])
                    #`(define (acc-id v)
                        (unless (pred-id v)
                          (error 'acc-id "~a is not a ~a struct" v 'id))
                        (vector-ref v ix))))))]))
; Now the same misuse gives a better error message:
> (our-struct "blah" ("blah" "blah"))

eval:74:0: our-struct: not an identifier

  at: "blah"

  in: (our-struct "blah" ("blah" "blah"))

Later, we’ll see how syntax-parse makes it even easier to +optional "guard" or "fender" expression. Instead of

[pattern template]

It can be:

[pattern guard template]

Let’s add a guard expression to our clause:

> (require (for-syntax racket/syntax))
> (define-syntax (our-struct stx)
    (syntax-case stx ()
      [(_ id (fields ...))
       ; Guard or "fender" expression:
       (for-each (lambda (x)
                   (unless (identifier? x)
                     (raise-syntax-error #f "not an identifier" stx x)))
                 (cons #'id (syntax->list #'(fields ...))))
       (with-syntax ([pred-id (format-id stx "~a?" #'id)])
         #`(begin
             ; Define a constructor.
             (define (id fields ...)
               (apply vector (cons 'id  (list fields ...))))
             ; Define a predicate.
             (define (pred-id v)
               (and (vector? v)
                    (eq? (vector-ref v 0) 'id)))
             ; Define an accessor for each field.
             #,@(for/list ([x (syntax->list #'(fields ...))]
                           [n (in-naturals 1)])
                  (with-syntax ([acc-id (format-id stx "~a-~a" #'id x)]
                                [ix n])
                    #`(define (acc-id v)
                        (unless (pred-id v)
                          (error 'acc-id "~a is not a ~a struct" v 'id))
                        (vector-ref v ix))))))]))
; Now the same misuse gives a better error message:
> (our-struct "blah" ("blah" "blah"))

eval:81:0: our-struct: not an identifier

  at: "blah"

  in: (our-struct "blah" ("blah" "blah"))

Later, we’ll see how syntax-parse makes it even easier to check usage and provide helpful messages about mistakes.

4.3 Using dot notation for nested hash lookups

The previous two examples used a macro to define functions whose names were made by joining identifiers provided to the macro. This example does the opposite: The identifier given to the macro is split into @@ -228,7 +253,7 @@ dictionaries that contain other dictionaries. In a jsexpr?< these are represented by nested hasheq tables.

JavaScript you can use dot notation:

foo = js.a.b.c;

In Racket it’s not so convenient:

; Nested hasheqs typical of a jsexpr:
> (define js (hasheq 'a (hasheq 'b (hasheq 'c "value"))))
; Typical annoying code to get something:
> (hash-ref (hash-ref (hash-ref js 'a) 'b) 'c)

"value"

We can write a helper function to make this a bit cleaner:

; This helper function:
> (define/contract (hash-refs h ks [def #f])
    ((hash? (listof any/c)) (any/c) . ->* . any)
    (with-handlers ([exn:fail? (const (cond [(procedure? def) (def)]
                                            [else def]))])
      (for/fold ([h h])
        ([k (in-list ks)])
        (hash-ref h k))))
; Lets us say:
> (hash-refs js '(a b c))

"value"

That’s not bad. Can we go even further and use a dot notation somewhat like JavaScript?

; This macro:
> (require (for-syntax racket/syntax))
> (define-syntax (hash.refs stx)
    (syntax-case stx ()
      [(_)
       (raise-syntax-error #f "Expected (hash.key0[.key1 ...] [default])"
                           stx #'chain)]
      [(_ chain)
       #'(hash.refs chain #f)]
      [(_ chain default)
       (unless (symbol? (syntax-e #'chain))
         (raise-syntax-error #f "Expected (hash.key0[.key1 ...] [default])"
                             stx #'chain))
       (let ([xs (map (lambda (x)
                        (datum->syntax stx (string->symbol x)))
                      (regexp-split #rx"\\."
                                    (symbol->string (syntax->datum #'chain))))])
         (unless (and (>= (length xs) 2)
                      (not (eq? (syntax-e (cadr xs)) '||)))
           (raise-syntax-error #f "Expected hash.key" stx #'chain))
         (with-syntax ([h (car xs)]
                       [ks (cdr xs)])
           #'(hash-refs h 'ks default)))]))
; Gives us "sugar" to say this:
> (hash.refs js.a.b.c)

"value"

It works!

We’ve started to appreciate that our macros should give helpful messages when used in error. We tried to do that here. Let’s -deliberately elicit various errors. Are the messages helpful?

> (hash.refs)

eval:80:0: hash.refs: Expected (hash.key0[.key1 ...]

[default])

  at: chain

  in: (hash.refs)

> (hash.refs 0)

eval:83:0: hash.refs: Expected (hash.key0[.key1 ...]

[default])

  at: 0

  in: (hash.refs 0 #f)

> (hash.refs js)

eval:84:0: hash.refs: Expected hash.key

  at: js

  in: (hash.refs js #f)

> (hash.refs js.)

eval:85:0: hash.refs: Expected hash.key

  at: js.

  in: (hash.refs js. #f)

Not too bad.

Maybe we’re not convinced that writing (hash.refs js.a.b.c) +deliberately elicit various errors. Are the messages helpful?

> (hash.refs)

eval:87:0: hash.refs: Expected (hash.key0[.key1 ...]

[default])

  at: chain

  in: (hash.refs)

> (hash.refs 0)

eval:90:0: hash.refs: Expected (hash.key0[.key1 ...]

[default])

  at: 0

  in: (hash.refs 0 #f)

> (hash.refs js)

eval:91:0: hash.refs: Expected hash.key

  at: js

  in: (hash.refs js #f)

> (hash.refs js.)

eval:92:0: hash.refs: Expected hash.key

  at: js.

  in: (hash.refs js. #f)

Not too bad.

Maybe we’re not convinced that writing (hash.refs js.a.b.c) is really clearer than (hash-refs js '(a b c)). Maybe we won’t actually use this approach. But the Racket macro system makes it a possible choice.

5 Syntax parameters

"Anaphoric if" or "aif" is a popular macro example. Instead of writing:

(let ([tmp (big-long-calculation)])
  (if tmp
      (foo tmp)
      #f))

You could write:

(aif (big-long-calculation)
     (foo it)
     #f)

In other words, when the condition is true, an it identifier