272 lines
8.8 KiB
Racket
272 lines
8.8 KiB
Racket
#lang scribble/doc
|
|
@(require "base.rkt")
|
|
|
|
@declare-exporting[rackunit #:use-sources (rackunit)]
|
|
|
|
@title[#:tag "internals"]{RackUnit Internals and Extension API}
|
|
|
|
This section describes RackUnit's facilities for customizing the
|
|
behavior of checks and tests and for creating new kinds of test
|
|
runners.
|
|
|
|
@section{Customizing Check Evaluation}
|
|
|
|
The semantics of checks are determined by the parameters
|
|
@racket[current-check-around] and
|
|
@racket[current-check-handler]. Other testing form such as
|
|
@racket[test-begin] and @racket[test-suite] change the value
|
|
of these parameters.
|
|
|
|
@defparam[current-check-handler handler (-> any/c any)]{
|
|
|
|
Parameter containing the function that handles exceptions
|
|
raised by check failures. The default value is @racket[raise].
|
|
}
|
|
|
|
@defparam[current-check-around check (-> (-> any) any)]{
|
|
|
|
Parameter containing the function that handles the execution
|
|
of checks. The default value wraps the evaluation of
|
|
@racket[thunk] in a @racket[with-handlers] call that calls
|
|
@racket[current-check-handler] if an exception is raised and then
|
|
(when an exception is not raised) discards the result, returning
|
|
@racket[(void)].
|
|
}
|
|
|
|
@section{Customizing Test Evaluation}
|
|
|
|
Just like with checks, there are several parameters that
|
|
control the semantics of compound testing forms.
|
|
|
|
@defparam[current-test-name name (or/c string? false/c)]{
|
|
|
|
This parameter stores the name of the current test case. A
|
|
value of @racket[#f] indicates a test case with no name,
|
|
such as one constructed by @racket[test-begin].
|
|
}
|
|
|
|
@defparam[current-test-case-around handler (-> (-> any) any)]{
|
|
|
|
This parameter handles evaluation of test cases. The value
|
|
of the parameter is a function that is passed a thunk (a
|
|
function of no arguments). The function, when applied,
|
|
evaluates the expressions within a test case. The default
|
|
value of the @racket[current-test-case-around] parameters
|
|
evaluates the thunk in a context that catches exceptions and
|
|
prints an appropriate message indicating test case failure.
|
|
}
|
|
|
|
@defproc[(test-suite-test-case-around [thunk (-> any)]) any]{
|
|
|
|
The @racket[current-test-case-around] parameter is
|
|
parameterized to this value within the scope of a
|
|
@racket[test-suite]. This function creates a test case
|
|
structure instead of immediately evaluating the thunk.
|
|
}
|
|
|
|
@defproc[(test-suite-check-around [thunk (-> any/c)]) any/c]{
|
|
|
|
The @racket[current-check-around] parameter is parameterized
|
|
to this value within the scope of a @racket[test-suite].
|
|
This function creates a test case structure instead of
|
|
immediately evaluating a check.
|
|
}
|
|
|
|
@;{--------}
|
|
|
|
@section[#:tag "running"]{Programmatically Running Tests and Inspecting Results}
|
|
|
|
RackUnit provides an API for running tests, from which
|
|
custom UIs can be created.
|
|
|
|
@subsection{Result Types}
|
|
|
|
@defstruct[(exn:test exn) ()]{
|
|
|
|
The base structure for RackUnit exceptions. You should
|
|
never catch instances of this type, only the subtypes
|
|
documented below.}
|
|
|
|
@defstruct[(exn:test:check exn:test) ([stack (listof check-info)])]{
|
|
|
|
A @racket[exn:test:check] is raised when an check fails, and
|
|
contains the contents of the check-info stack at the
|
|
time of failure.}
|
|
|
|
@defstruct[test-result ([test-case-name (or/c string #f)])]{
|
|
|
|
A test-result is the result of running the test with
|
|
the given name (with @racket[#f] indicating no name is available).}
|
|
|
|
@defstruct[(test-failure test-result) ([result any])]{
|
|
|
|
Subtype of test-result representing a test failure.}
|
|
|
|
@defstruct[(test-error test-result) ([result exn])]{
|
|
|
|
Subtype of test-result representing a test error.}
|
|
|
|
@defstruct[(test-success test-result) ([result any])]{
|
|
|
|
Subtype of test-result representing a test success.}
|
|
|
|
|
|
@subsection{Functions to Run Tests}
|
|
|
|
@defproc[(run-test-case (name (or/c string #f)) (action (-> any)))
|
|
test-result]{
|
|
|
|
Runs the given test case, returning a result representing success,
|
|
failure, or error.
|
|
}
|
|
|
|
|
|
@defproc[(run-test (test (or/c test-case? test-suite?)))
|
|
(flat-murec-contract ([R (listof (or/c test-result? R))]) R)]{
|
|
|
|
Runs the given test (test case or test suite) returning a
|
|
tree (list of lists) of results}
|
|
|
|
Example:
|
|
|
|
@racketblock[
|
|
(run-test
|
|
(test-suite
|
|
"Dummy"
|
|
(test-case "Dummy" (check-equal? 1 2))))
|
|
]
|
|
|
|
@defproc[(fold-test-results [result-fn ('b 'c ... 'a . -> . 'a)]
|
|
[seed 'a]
|
|
[test (or/c test-case? test-suite?)]
|
|
[#:run run (string (() -> any) . -> . 'b 'c ...)]
|
|
[#:fdown fdown (string 'a . -> . 'a)]
|
|
[#:fup fup (string 'a . -> . 'a)])
|
|
'a]{
|
|
|
|
Fold @racket[result-fn] pre-order left-to-right depth-first
|
|
over the results of @racket[run]. By default @racket[run]
|
|
is @racket[run-test-case] and @racket[fdown] and
|
|
@racket[fup] just return the seed, so @racket[result-fn] is
|
|
folded over the test results.
|
|
|
|
This function is useful for writing custom folds (and hence UIs) over
|
|
test results without you having to take care of all the expected setup
|
|
and teardown. For example, @racket[fold-test-results] will run test
|
|
suite before and after actions for you. However it is still flexible
|
|
enough, via its keyword arguments, to do almost anything that
|
|
@racket[foldts-test-suite] can. Hence it should be used in preference to @racket[foldts-test-suite].
|
|
|
|
The @racket[result-fn] argument is a function from the results of
|
|
@racket[run] (defaults to a @racket[test-result]) and the seed to a
|
|
new seed.
|
|
|
|
The @racket[seed] argument is any value.
|
|
|
|
The @racket[test] argument is a test case or test suite.
|
|
|
|
The @racket[run] argument is a function from a test case name (string)
|
|
and action (thunk) to any values. The values produced by @racket[run]
|
|
are fed into the @scheme[result-fn].
|
|
|
|
The @racket[fdown] argument is a function from a test suite name
|
|
(string) and the seed, to a new seed.
|
|
|
|
The @racket[fup] argument is a function from a test suite name
|
|
(string) and the seed, to a new seed.
|
|
}
|
|
|
|
Examples:
|
|
|
|
The following code counts the number of successes:
|
|
|
|
@racketblock[
|
|
(define (count-successes test)
|
|
(fold-test-results
|
|
(lambda (result seed)
|
|
(if (test-success? result)
|
|
(add1 seed)
|
|
seed))
|
|
0
|
|
test))]
|
|
|
|
The following code returns the symbol @racket['burp] instead
|
|
of running test cases. Note how the @racket[result-fn] receives the
|
|
value of @racket[run].
|
|
|
|
@racketblock[
|
|
(define (burp test)
|
|
(fold-test-results
|
|
(lambda (result seed) (cons result seed))
|
|
null
|
|
test
|
|
#:run (lambda (name action) 'burp)))]
|
|
|
|
|
|
@defproc[(foldts-test-suite [fdown (test-suite string thunk thunk 'a -> 'a)]
|
|
[fup (test-suite string thunk thunk 'a 'a -> 'a)]
|
|
[fhere(test-case string thunk 'a -> 'a)]
|
|
[seed 'a]
|
|
[test (or/c test-case? test-suite?)])
|
|
'a]{
|
|
|
|
The @racket[foldts-test-suite] function is a nifty tree fold (created by Oleg
|
|
Kiselyov) that folds over a test in a useful way
|
|
(@racket[fold-test-results] isn't that useful as you can't specify
|
|
actions around test cases).
|
|
|
|
The @racket[fdown] argument is a function of test suite, test suite
|
|
name, before action, after action, and the seed. It is run when a
|
|
test suite is encountered on the way down the tree (pre-order).
|
|
|
|
The @racket[fup] argument is a function of test suite, test suite
|
|
name, before action, after action, the seed at the current level, and
|
|
the seed returned by the children. It is run on the way up the tree
|
|
(post-order).
|
|
|
|
The @racket[fhere] argument is a function of the test case, test case
|
|
name, the test case action, and the seed. (Note that this might change
|
|
in the near future to just the test case. This change would be to
|
|
allow @racket[fhere] to discriminate subtypes of test-case, which in
|
|
turn would allow test cases that are, for example, ignored).
|
|
}
|
|
|
|
Example:
|
|
|
|
Here's the implementation of @racket[fold-test-results] in terms of
|
|
@racket[foldts-test-suite]:
|
|
|
|
@racketblock[
|
|
(define (fold-test-results suite-fn case-fn seed test)
|
|
(foldts-test-suite
|
|
(lambda (suite name before after seed)
|
|
(before)
|
|
(suite-fn name seed))
|
|
(lambda (suite name before after seed kid-seed)
|
|
(after)
|
|
kid-seed)
|
|
(lambda (case name action seed)
|
|
(case-fn
|
|
(run-test-case name action)
|
|
seed))
|
|
seed
|
|
test))
|
|
]
|
|
|
|
If you're used to folds you'll probably be a bit surprised that the
|
|
functions you pass to @racket[foldts-test-suite] receive both the structure they
|
|
operate on, and the contents of that structure. This is indeed
|
|
unusual. It is done to allow subtypes of test-case and test-suite to
|
|
be run in customised ways. For example, you might define subtypes of
|
|
test case that are ignored (not run), or have their execution time
|
|
recorded, and so on. To do so the functions that run the test cases
|
|
need to know what type the test case has, and hence is is necessary to
|
|
provide this information.
|
|
|
|
If you've made it this far you truly are a master RackUnit hacker. As
|
|
a bonus prize we'll just mention that the code in
|
|
@racketfont{hash-monad.rkt} and @racketfont{monad.rkt} might be of
|
|
interest for constructing user interfaces. The API is still in flux,
|
|
so isn't documented here. However, do look at the implementation of
|
|
@racket[run-tests] for examples of use.
|