suzanne.soy/racket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
e4add14942
racket/collects/handin-server/doc.txt
Eli Barzilay c88354bb17 small fixes 
svn: r1295
2005-11-12 16:08:46 +00:00

896 lines
40 KiB
Plaintext
Raw Blame History

_Handin Server and Client_
The "handin-server" directory contains a server to be run by a course
instructor for accepting homework assignments and reporting on
submitted assignments.
The "handin-client" directory contains a client to be customized then
re-distributed to students in the course. The customized client will
embed a particular hostname and port where the server is running, as
well as a server certificate.
With a customized client, students simply install a ".plt" file --- so
there's no futzing with configuration dialogs and certificates. A
student can install any number of clients at once (assuming that the
clients are properly customized, as described below).
The result, on the student's side, is a "Handin" button in DrScheme's
toolbar. Clicking the "Handin" button allows the student to type a
password and upload the current content of the definitions and
interactions window to the course instructor's server. The "File" menu
is also extended with a "Manage..." menu item for managing a handin
account (i.e., changing the password and other information, or
creating a new account if the instructor configures the server to
allow new accounts). Students can submit joint work by submitting with
a concatenation of usernames separated by a "+".
On the instructor's side, the handin server can be configured to check
the student's submission before accepting it.
The handin process uses SSL, so it is effectively as secure as the
server and each user's password.
Quick Start for a Test Drive:
============================================
1. Create a new directory.
2. Copy "server-cert.pem" from the "handin-client" collection
to the new directory.
NOTE: for real use, you need a new certificate.
NOTE: see also "Where is the collection?", below
3. Copy "private-key.pem" from the "handin-server" collection
to the new directory.
NOTE: for real use, you need a new key.
4. Create a file "users.ss" with the following content:
((tester ("8fe4c11451281c094a6578e6ddbf5eed"
"Chester Tester"
"123")))
5. Make an "active" subdirectory in your new directory.
6. Make a "test" subdirectory in "active".
7. In your new directory, run
mred -mvqM handin-server
8. In the "handin-client" collection, edit "info.ss" and
uncomment the line
(define server:port "localhost:7979")
9. Start DrScheme, click "Handin" to run the client, submit with
username "tester" and password "pw".
The submitted file will be .../active/test/tester/handin.scm.
10. Check the status of your submission by pointing a web browser at
https://localhost:7980/servlets/status.ss
Note the "s" in "https". Use the "tester" username and "pw"
password, as before.
-------------------------------------------------------------------
| "Where is the collection?" |
|- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -|
| If you obtained the server and client by installing a .plt file, |
| then the "handin-server" and "handin-client" directories |
| are likely in your PLT addon space: |
| Windows: |
| %USERPROFILE%\Application Data\PLT Scheme\<version>\collects |
| Unix: |
| ~/.plt-scheme/<version>/collects |
| Mac OS X: |
| ~/Library/PLT Scheme/<version>/collects |
-------------------------------------------------------------------
Client Customization
============================================
To customize the client:
1. Rename (or make a copy of) the "handin-client" collection
directory. The new name should describe your class uniquely.
For example, "uu-cpsc2010" is a good name for CPSC 2010
at the University of Utah.
2. Edit the first three definitions of "info.ss" in your renamed
client collection:
* For `name', choose a name for the handin tool as it will
appear in DrScheme's interface (e.g., the "XXX" for the
"Manage XXX Handin Account..." menu item). Again, make the
name specific to the course, in case a student installs
multiple handin tools. Do not use "Handin" as the last part
of the name, since "Handin" is always added for button and
menu names.
* For `collection', use the name that you chose for your
collection directory (i.e., whatever you changed
"handin-client" to).
* For `server:port', uncomment the line, and use the hostname
and port where the server will be running to accept handin
submissions.
Optionally uncomment and edit the next two definitions,
`web-menu-name' and `web-address', to add an item to the "Help"
menu that opens a (course-specific) web page.
3. Replace "icon.png" in your renamed directory with a new 32x32
icon. This icon is displayed on startup with DrScheme's splash
screen, and it is included at half size on the "Handin"
button. Again, choose a distinct icon for the benefit of
students who install multiple handin tools.
4. Replace "server-cert.pem" in your renamed directory with a
server certificate. The file "server-cert.pem" in
"handin-client" collection is ok for testing, but the point of
this certificate is to make handins secure, so you should
generate a new (self-certifying) certificate and keep its key
private. (See server setup, below.)
5. Run
mzc --collection-plt <name>.plt <name>
where <name> is the name that you chose for your directory
(i.e., whatever you changed "handin-client" to).
6. Distribute <name>.plt to students for installation into their
copies of DrScheme. The students need not have access to the
DrScheme installation directory; the tool will be installed on
the filesystem in the student's personal space. If you want to
install it once on a shared installation, use setup-plt with the
--all-users flag.
Server Setup
============================================
The server must be run from a directory that is specially prepared to
host the server. This directory contains the following files and
sub-directories:
* "server-cert.pem" --- the server's certificate. To create a
certificate and key with openssl:
openssl req -new -nodes -x509 -days 365 -out server-cert.pem
-keyout private-key.pem
* "private-key.pem" --- the private key to go with "server-cert.pem".
Whereas "server-cert.pem" gets distributed to students with the
handin client, "private-key.pem" is kept private.
* "config.ss" (optional) --- configuration options. The file format
is
((<key> <val>) ...)
for the following keys:
'port-number : the port for the main handin server; the default
is 7979
'https-port-number : the port for the handin-status HTTPS
server; the default is one more than the main server's port
(so the transitive default is 7980)
'session-timeout : number of seconds before the session
times-out -- the client is given this many seconds for the
login stage and then starts again so the same number of
seconds is given for the submit-validation process; the
default is 300
'session-memory-limit : maximum size in bytes of memory allowed
for per-session computation, if per-session limits are
supported (i.e., when using MrEd3m and MzScheme3m with memory
accounting); the default is 40000000
'default-file-name : the default filename that will be saved
with the submission contents. The default is "handin.scm".
'max-upload : maximum size in bytes of an acceptable submission;
the default is 500000
'max-upload-keep : maximum index of submissions to keep; the
most recent submission is "handin.scm" (by default), the next
oldest is in "BACKUP-0/handin.scm", next oldest is
"BACKUP-1/handin.scm", etc.; the default is 9
'user-regexp : a regular expression that is used to validate
usernames; alternatively, this can be #f meaning no
restriction, or a list of permitted strings. Young students
often choose exotic usernames that are impossible to
remember, and forget capitalization, so the default is fairly
strict: #rx"^[a-z][a-z0-9]+$"; a "+" is always disallowed in
a username, since it is used in a submission username to
specify joint work
'user-desc : a plain-words description of the acceptable
username format (according to user-regexp above); #f stands
for no description; the default is "alphanumeric string"
which matches the default user-regexp
'username-case-sensitive? : a boolean; when #f, usernames
are case-folded for all purposes; defaults to #f (note that
you should not set this to #t on Windows or when using other
case-insensitive filesystems, since usernames are used as
directory names)
'allow-new-users : a boolean indicating whether to allow
new-user requests from a client tool; the default is #f
'allow-change-info : a boolean indicating whether to allow
changing user information from a client tool (changing
passwords is always possible); the default is #f
'master-password : a string for an MD5 hash for a password that
allows login as any user; the default is #f, which disables
the password
'web-base-dir : if #f (the default), the built-in web server
will use the "status-web-root" in this collection for its
configuration; to have complete control over the built in
server, you can copy and edit "status-web-root" to the
directory you're running the handin server server from, and
add this configuration entry with the name of your new copy
(relative to the handin server directory)
'extra-fields : a list that describes extra fields of (string)
information for students; each element in this list is a list
of three values -- the name of the field, the regexp (or #f,
or a list of permitted string values), and a plain-words
description of acceptable strings. The default is
'(("Full Name" #f #f)
("ID#" #f #f)
("Email" #rx"^[^@<>\"`',]+@[a-zA-Z0-9_.-]+[.][a-zA-Z]+$"
"a valid email address"))
You can set this to a list of fields that you are interested
in keeping, for example:
'(("Full Name"
#rx"^[A-Z][a-zA-Z]+(?: [A-Z][a-zA-Z]+)+$"
"full name, no punctuations, properly capitalized")
("Utah ID Number"
#rx"^[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]$"
"Utah ID Number with exactly nine digits")
("Email"
#rx"^[^@<>\"`',]+@cs\\.utah\\.edu$"
"A Utah CS email address"))
The order of these fields will be used both on the client GUI
side and in the "users.ss" file (see below).
* "users.ss" (created if not present if a user is added) --- keeps
the list of user accounts, along with the associated password
(actually the MD5 hash of the password), and extra string fields
as specified by the 'extra-fields configuration entry (in the same
order). The file format is
((<username-sym> (<pw-md5-str> <extra-field> ...))
...)
For example, the default 'extra-field setting will make this:
((<username-sym> (<pw-md5-str> <full-name> <id> <email>))
...)
Username that begin with "solution" are special. They are used by
the HTTPS status server. Independent of the 'user-regexp and
'username-case-sensitive? configuration items, usernames are not
allowed to contain characters that are illegal in Windows
pathnames, they cannot end or begin in spaces or periods.
If the 'allow-new-users configuration allows new users, the
"users.ss" file can be updated by the server with new users. It
can always be updated by the server to change passwords.
If you have access to a standard Unix password file (from
"/etc/passwd" or "/etc/shadow"), then you can construct a
"users.ss" file that will allow users to use their normal
passwords. To achieve this, use a list with 'unix as the first
element and the system's encrypted password string as the second
element. Such passwords can be used, but when users change them,
a plain md5 hash will be used.
You can combine this with other fields from the password file to
create your "users.ss", but make sure you have information that
matches your 'extra-fields specification. For example, given this
system file:
foo:wRzN1u5q2SqRD:1203:1203:Foo Moo:/home/foo:/bin/tcsh
bar:$1$dKlU0OkJ$t63NU/eTzKz:1205:1205:Bar Z. Lie:/home/bar:/bin/bash
you can create a "users.ss" file as
((foo ((unix "wRzN1u5q2SqRD") "Foo Moo" "?"))
(bar ((unix "$1$dKlU0OkJ$t63NU/eTzKz") "Bar Z. Lie" "?")))
which can be combined with this setting for 'extra-fields in your
"config.ss":
...
(extra-fields (("Full Name" #f #f)
("TA" '("Alice" "Bob") "Your TA")))
...
and you can tell your students to use their department username
and password, and use the "Manage ..." dialog to properly set
their TA name.
* "active/" --- sub-directory for active assignments. A list of
active assignments is sent to a client tool when a student clicks
"Handin", based on the contents of this directory. The student
then selects from the list. The assignments are ordered in the
student's menu using `string<?', and the first assignment is the
default selection.
Within each directory, the student id is used for a sub-directory
name. Within each student sub-directory are directories for handin
attempts and successes. If a directory "ATTEMPT" exists, it
contains the most recent (unsuccessful) handin attempt.
Directories "SUCCESS-n" (where n counts from 0) contain successful
handins; the lowest numbered such directory represents the latest
handin.
A cleanup process in the server copies successful submission to
the student directory -- one level up from the corresponding
"SUCCESS-n" directory. This is done only for files and
directories that are newer in "SUCCESS-n" than in the submission
root, other files and directories are left intact. If external
tools add new content to the student directory (eg, a "grade"
file, as described below) it will stay there. If the machine
crashes or the server is stopped, the cleanup process might not
finish. When the server is started, it automatically runs the
cleanup process for each student directory.
Within a student directory, a file "handin.scm" (or some other
name if `default-file-name' is set) contains the actual
submission. A `checker' procedure can change this default file
name, and it can create additional files in an "ATTEMPT" directory
(to be copied by the cleanup process); see below on "checker.ss"
for more details.
For submissions from a normal DrScheme frame, a submission file
contains a copy of the student's definitions and interactions
windows. The file is in a binary format (to support non-text
code), and opening the file directly in DrScheme shows the
definitions part. To get both the definitions and interactions
parts, the file can be parsed with `unpack-submission' from
"utils.ss" (see below).
To submit an assignment as a group, students use a concatenation
of usernames separated by "+" and any number of spaces (e.g.,
"user1+user2"). The same syntax ("user1+user2") is used for the
directory for shared submissions, and the usernames are always
sorted so that the directory name is deterministic. Multiple
submissions for a particular user in different groups will be
rejected.
* "inactive/" --- sub-directory for inactive assignments, used by
the HTTPS status web server.
* "active/<assignment>/checker.ss" (optional) --- a module that
exports a `checker' function. This function receives two strings.
The first is a username list and the second is the submission as a
byte string. (See also `unpack-submission', etc. from "util.ss",
below.) To reject the submission, the `checker' function can
raise an exception; the exception message will be relayed back to
the student. The module is loaded when the current directory is
the main server directory, so it can read information from
"config.ss".
The first argument is a list of usernames with at least one
username, and more than one if this is a joint submission (where
the submission username was a concatenation of usernames separated
by "+").
The `checker' function is called with the current directory as
"active/<assignment>/<username(s)>/ATTEMPT", and the submission is
saved in the file "handin", and the timeout clock is reset to the
value of the 'session-timeout configuration. The checker function
can change "handin", and it can create additional files in this
directory. (Extra files in the current directory will be
preserved as it is later renamed to "SUCCESS-0", and copied to the
submission's root ("active/<assignment>/<user/s>/"), etc.) To
hide generated files from the HTTPS status web server interface,
put the files in a subdirectory, which is preserved but hidden
from the status interface.
The checker should return a string, such as "handin.scm", to use
in naming the submission file.
Alternatively, the module can bind `checker' to a list of three
procedures: a pre-checker, a checker, and a post-checker. All
three are applied in exactly the same way as the checker (same
arguments, and always within the submission directory), except
that:
- If there was an error during the pre-checker, and the submission
directory does not have a "SUCCESS-*" directory, then the whole
submission directory is removed. This is useful for checking
that the user/s are valid -- if you allow a submission only when
`users' is '("foo" "bar"), and "foo" tries to submit alone, then
the submission directory for "foo" should be removed to allow a
proper submission later. Note that the timeout clock is reset
only once, before the pre-checker is used.
- The post-checker is used at the end of the process, after the
"ATTEMPT" directory was renamed to "SUCCESS-0". At this stage,
the submission is considered successful, so this function should
avoid throwing an exception (it can, but the submission will
still be in place). This is useful for things like notifying
the user of the successful submission (see `message' below), or
sending a `receipt' email.
To specify only pre/post-checker, use #f for the one you want to
omit.
* "log.ss" (created if not present, appended otherwise) --- records
connections and actions, where each entry is of the form
(id time-str msg-str)
and `id' is an integer representing the connection (numbered
consecutively from 1 when the server starts) or 0 for a message
for server without a connection.
* "web-status-log.ss" (created if not present, appended otherwise)
--- records accesses of the HTTPS status web server.
* "[in]active/<assignment>/<user>/<filename>" (if submitted) --- the
most recent submission for <assignment> by <user> where <filename>
was returned by the checker (or the value of the
`default-file-name' configuration option if there's no checker).
If the submission is from multiple users, then "<user>" is
actually "<user1>+<user2>" etc. Also, if the cleanup process was
interrupted (by a machine failure, etc.), the submission may
actually be in "SUCCESS-n" as described above.
* "[in]active/<assignment>/<user>/grade" (optional) --- <user>'s
grade for <assignment>, to be reported by the HTTPS status web
server
* "[in]active/<assignment>/solution*" --- the solution to the
assignment, made available by the status server to any user who
logs in. The solution can be either a file or a directory with a
name that begins with "solution". In the first case, the status
web server will have a "Solution" link to the file, and in the
second case, all files in the "solution*" directory will be listed
and accessible.
The server can be run within either MzScheme or MrEd, but "utils.ss"
requires MrEd (which means that `checker' modules will likely require
the server to run under MrEd).
The server currently provides no mechanism for a graceful shutdown,
but terminating the server is no worse than a network outage. (In
particular, no data should be lost.) To reconfigure the server (e.g.,
to change the set of active assignments), stop it and restart it.
The client and server are designed to be robust against network
problems and timeouts. The client-side tool always provides a "cancel"
button for any network transaction. For handins, "cancel" is
guaranteed to work up to the point that the client sends a "commit"
command; this command is sent only after the server is ready to record
the submission (having run it through the checker, if any), but before
renaming "ATTEMPT". Also, the server responds to a commit with "ok"
only after it has written the file. Thus, when the client-side tool
reports that the handin was successful, the report is
reliable. Meanwhile, the tool can also report successful cancels most
of the time. In the (normally brief) time between a commit and an "ok"
response, the tool gives the student a suitable warning that the
cancel is unreliable.
To minimize human error, the number of active assignments should be
limited to one whenever possible. When multiple assignments are
active, design a checker to help ensure that the student has selected
the correct assignment in the handin dialog.
A student can download his/her own submissions through a web server
that runs concurrently with the handin server. The starting URL is
https://SERVER:PORT/servlets/status.ss
to obtain a list of all assignments, or
https://SERVER:PORT/servlets/status.ss?handin=ASSIGNMENT
to start with a specific assignment (named ASSIGNMENT). The default
PORT is 7980.
Checker Utilities
============================================
The _utils.ss_ module provides utilities helpful in implementing
`checker' functions:
> (unpack-submission bytes) - returns two text% objects corresponding
to the submitted definitions and interactions windows.
> (make-evaluator language teachpack-paths program-port) - returns a
function of one required argument for evaluating expressions in the
designated language, and loading teachpacks that are specified in
`teachpack-paths'. The `program-port' is an input port that
produces the content of the definitions window; use
`(open-input-string "")' for an empty definitions window.
The `language' can be:
* a symbol indicating a built-in language (currently, only
'mzscheme), or a teaching language -- one of 'beginner,
'beginner-abbr, 'intermediate, 'intermediate-lambda, or 'advanced.
* a list that begins with a 'lib symbol stands for the language
defined by this (quoted) module specification.
* a list that begins with a 'begin symbol means that the code will
not be evaluated in a module context at all, it will simply be
evaluated in a new namespace, after evaluating this list.
The `teachpack-paths' list specifies additional code to load, can be
one of:
* paths to teachpacks to load into the module.
* a list that begins with a 'begin symbol is arbitrary code that is
prefixed into the submitted program.
The actual evaluation of expressions happens in a newly created
eventspace and namespace, and under the supervision of a strict
security guard that reading files only from PLT collections, and no
other operations.
Additional arguments to the evaluator function are special messages
that retrieve additional information. Currently, only
'execute-counts is used (see below).
> (make-evaluator/submission language teachpack-paths bytes) - like
`make-evaluator', but the definitions content is supplied as a
submission byte string. The byte string is opened for reading, with
line-counting enabled.
> (call-with-evaluator language teachpack-paths program-port proc) -
calls `proc' with an evaluator for the given language, teachpack
paths, and initial definition content as supplied by a port. It also
sets the current error-value print handler to print values in a way
suitable for `lang', it initializes `current-run-status' with
"executing your code", and it catches all exceptions to re-raise
them in a form suitable as a submission error.
> (call-with-evaluator/submission language teachpack-paths bytes proc) -
like `call-with-evaluator', but the definitions content is supplied
as a submission string. The byte string is opened for reading,
with line-counting enabled.
> (evaluate-all source input-port eval) - like `load' on an input
port.
> (evaluate-submission bytes eval) - like `load' on a non-test-suite
submission byte string.
> coverage-enabled - parameter that controls whether coverage testing
is enabled. If it set to true, the errortrace collection will be
used to collect coverage information during evaluation of the
submission, this information is collected before additional
checker-evaluations. To retrieve the collected information, apply
the evaluation function with a second argument of 'execute-counts
(the first argument will be ignored). The resulting value is the
same as the result of errortrace's `get-execute-counts', with all
non-submission entries filtered out.
> (check-proc eval expect-v compare-proc proc-name arg ...) - calls
the function named `proc-name' using the evaluator `eval', giving it
the (unquoted) arguments `arg'... Let `result-v' be the result of
the call; unless `(compare-proc result-v expect-v)' is true, an
exception is raised.
Every exception or result mismatch during the call to `check-proc'
phrased suitably for the handin client.
> (check-defined eval name) - checks whether `name' is defined in the
evaluator `eval', and raises an error if not (suitably phrased for
the handin client). If it is defined as non-syntax, its value is
returned. Warning: in the beginner language level, procedure
definitions are bound as syntax.
> (look-for-tests text name n) - inspects the given text% object to
determine whether it contains at least `n' tests for the function
`name'. The tests must be top-level expressions.
> (user-construct eval name arg ...) - like `check-proc', but with no
result checking. This function is often useful for calling a
student-defined constructor.
> test-history-enabled - parameter that controls how run-time errors
are reported to the handin client. If the parameter's value is true,
then the complete sequence of tested expressions is reported to the
handin client for any test failure. Set this parameter to true when
testing programs that use state.
> (message string [styles]) - if given only a string, this string will
be shown on the client's submission dialog; if `styles' is also
given, it can be the symbol 'final, which will be used as the text
on the handin dialog after a successful submission instead of
"Handin successful." (useful for submissions that were saved, but
had problems); finally, `styles' can be used as a list of styles for
a `message-box' dialog on the client side, and the resulting value
is returned as the result of `message'. You can use that to send
warnings to the student and wait for confirmation.
> (current-run-status string-or-#f) - registers information about the
current actions of the checker, in case the session is terminated
due to excessive memory consumption. For example, a checker might
set the status to indicate which instructor-supplied test was being
executed when the session ran out of memory. This status is only
used when per-session memory limits are supported (i.e., under
MrEd3m or MzScheme3m with memory accounting), but in both cases, a
string value will also be passed on to `message' above.
> (current-value-printer proc) - a parameter that controls how values
are printed, a procedure that expects a Scheme value and returns a
string representation for it. The default value printer uses
pretty-print, with DrScheme-like settings.
> (reraise-exn-as-submission-problem thunk) - calls thunk in a context
that catches exceptions and re-raises them in a form suitable as a
submission error.
> (LOG fmt args ...) - produces a line in the server log file, using
the given format string and arguments.
> (timeout-control msg) - control the timeout for this session. The
timeout is initialized by the value of the 'session-timeout
configuration entry, and the checker can use this procedure to
further control it: if msg is 'reset the timeout is reset to
'session-timeout seconds; if msg is a number the timeout will be set
to that many seconds in the future. The timeout can be completely
disabled by (timeout-control #f). (Note that before the checker is
used (after the pre-checker, if specified), the timer will be reset
to the 'session-timeout value.)
Extra Checker Utilities
============================================
The _extra-utils.ss_ module provides a higher-level of utilities,
helpful in implementing `checker' functions that are intended for a
more automated system. This module is a language module -- a typical
checker that uses it looks like this:
(module checker (lib "extra-utils.ss" "handin-server")
(check: :language 'intermediate
:users pairs-or-singles-with-warning
:coverage? #t
(!procedure Fahrenheit->Celsius 1)
(!test (Fahrenheit->Celsius 32) 0)
(!test (Fahrenheit->Celsius 212) 100)
(!test (Fahrenheit->Celsius -4) -20)
...
(!all-covered)))
> (check: :key val ... body ...)
Construct a checker procedure.
The `check:' macro will construct an appropriate checker function,
using keywords for features that you want, the body of the checker can
contain arbitrary code, using all utilities from "utils.ss" (see
above), as well as additional ones (see below).
Keywords for configuring `check:':
* :users -- specification of users that are acceptable for
submission. Can be either a list of user lists, each representing a
known team, or procedure which will accept a list of users and throw
an exception if they are unacceptable. The default is to accept
only single-user submissions. `pairs-or-singles-with-warning' is a
useful value for pair submission where the pairs are unknown (see
below).
* :eval? -- whether submissions should be evaluated. Defaults to #t.
Note that if it is specified as #f, then the checker body will not
be able to run any tests on the code, unless it contains code that
performs some evaluation (eg, using the facilities of "utils.ss").
* :language -- the language that is used for evaluating submissions,
same as the `language' argument for `make-evaluator' (see above).
There is no default for this, so it must be set or an error is
raised.
* :teachpacks -- teachpacks for evaluating submissions, same as the
`teachpacks' argument for `make-evaluator' (see above). Defaults to
null -- no teachpacks.
* :create-text? -- if true, then a textual version of the submission
is saved as `text.scm' in a `grading' subdirectory. This is
intended for printouts and grading, and is in a subdirectory so
students will not see it on the status web server. Defaults to #t.
* :textualize? -- if true, then all submissions are converted to text,
trying to convert objects like comment boxes and test cases to some
form of text. Defaults to #f, meaning that an exception is raised
for submissions that are not all text.
* :maxwidth -- a number that specifies maximum line lengths for
submissions (a helpful feature for reading student code). Defaults
to 79. This feature can be disabled if set to #f.
* :output -- the name of the original handin file (unrelated to the
text-converted files). Defaults to "hw.scm".
* :student-line -- when a submission is converted to text, it begins
with lines describing the students that have submitted it; this is
used to specify the format of these lines. It is a string with
holes that that `user-substs' (see below) fills out. The default is
"Student: {username} ({Full Name} <{Email}>)", which requires a
"Full Name" and "Email" entries in the server's extra-fields
configuration. These lines are always prefixed with ";;> ".
* :extra-lines -- a list of lines to add after the student lines, all
with a ";;> " prefix too. Defaults to a single line: "Maximum
points for this assignment: <+100>".
* :user-error-message -- a string that is used to report an error that
occurred during evaluation of the submitted code (not during
additional tests). It can be a plain string which will be used as
the error message, or a string with single a "~a" (or "~e", "~s",
"~v") that will be used as a format string with the actual error
message. The default is "Error in your code --\n~a". Examples of
these:
"there is an error in your program, hit \"Run\" and debug your code"
"There is an error in your program:\n----\n~a\n----\n
Hit \"Run\" and debug your code."
Alternatively, the value can be a procedure that will be invoked
with the error message. The procedure can do anything it wants, and
if it does not raise an exception, then the checker will proceed as
usual. For example:
(lambda (msg)
(add-header-line! "Erroneous submission!")
(add-header-line! (format " --> ~a" msg))
(message (string-append
"You have an error in your program -- please hit"
" \"Run\" and debug your code.\n"
"Email the course staff if you think your code is fine.\n"
"(The submission has been saved but marked as erroneous.)")
'(ok))
(message "Handin saved as erroneous." 'final))
(Note that if you do this, then additional tests should be adjusted
to not raise an exception too.)
* :value-printer -- if specified, this will be used for
`current-value-printer' (see above).
* :coverage? -- collect coverage information when evaluating the
submission (not including additional checker tests). This is needed
for the `!all-covered' procedure below. Does not work with
non-textual submissions.
Within the body of `check:', `users' and `submission' will be bound to
the checker arguments -- a (sorted) list of usernames and the
submission as a byte string. In addition to the functionality below,
you can use `((submission-eval) expr)' to evaluate expressions in the
submitted code context, and you can use `(with-submission-bindings (id
...) body ...)' to evaluate the body when `id's are bound to their
value from the submission code.
> (pre: body ...)
> (post: body ...)
These two macros define a pre- and a post-checker. In their body,
`users' and `submission' are bound as in `check:', but there is
nothing else special about these. See the description of the
`pre-checker' and `post-checker' values for what can be done with
these, and note that the check for valid users is always first. An
example for a sophisticated `post:' block is below -- it will first
disable timeouts for this session, then it will send a email with a
submission receipt, with CC to the TA (assuming a single TA), and
pop-up a message telling the student about it:
(require (lib "sendmail.ss" "net"))
(post:
(define info
(format "hw.scm: ~a ~a"
(file-size "hw.scm")
(file-or-directory-modify-seconds "hw.scm")))
(timeout-control 'disable)
(LOG "Sending a receipt: ~a" info)
(send-mail-message
"course-staff@university.edu"
"Submission Receipt"
(map (lambda (user) (user-substs user "{Full Name} <{Email}>"))
users)
(list (user-substs (car users) "{TA Name} <{TA Email}>"))
null
`("Your submission was received" ,info))
(message (string-append
"Your submission was successfully saved."
" You will get an email receipt within 30 minutes;"
" if not, please contact the course staff.")
'(ok)))
> submission-eval
A parameter that holds an evaluation procedure for evaluating code
in the submission context.
> (user-data user)
Returns a user information given a username. The returned
information is a list of strings that corresponds to the configured
`extra-fields' (see above).
> (user-substs user str)
Given a username, this procedure will lookup the user's extra-fields
information (see above) and then substitute field names in {braces}
by the corresponding value. An error will be signaled if a field
name is missing. Also, "{username}" will always be replaced by the
username.
This is used to process the `:student-line' value in the checker,
but it is provided for additional uses. See the above sample code
for `post:' for using this procedure.
> (pairs-or-singles-with-warning users)
This procedure is intended for use as the :users entry in a checker.
It will do nothing if there are two users, and throw an error if
there are more. If there is a single user, then the user will be
asked to verify a single submission -- if the students cancels, then
an exception is raised so the submission directory is retracted. If
the student approves this, the question is not repeated (this is
marked by creating a directory with a known name). This is useful
for cases where you want to allow free pair submissions -- students
will often try to submit their work alone, and later on re-submit
with a partner.
> (teams-in-file team-file)
This procedure *returns* a procedure that can be used for the :users
entry in a checker. The team file (relative from the server's main
directory) is expected to have user entries -- a sequence of
s-expressions, each one a string or a list of strings. The
resulting procedure will allow submission only by teams that are
specified in this file. Furthermore, if this file is modified, the
new contents will be used immediately, so there is no need to
restart the server of you want to change student teams. (But
remember that if you change ("foo" "bar") to ("foo" "baz"), and
there is already a "bar+foo" submission directory, then the system
will not allow "foo" to submit with "bar".)
> (add-header-line! line)
During the checker operation, this procedure can be used to add
header lines to the text version of the submitted file. It will not
have an effect if `:create-text?' is false.
> (procedure/arity? proc arity)
Returns #t if `proc' is a procedure that accepts `arity' arguments.
> (!defined id ...)
A macro that checks that the given identifiers are defined in the
(evaluated) submission, and throws an error otherwise.
> (!procedure id [arity])
> (!procedure* expr [arity])
`!procedure' checks that `id' is defined, and is bound to a
procedure; `!*procedure' is similar but omits the defined check,
making it usable with any expression (which is evaluated in the
submission context).
> (!integer id)
> (!integer* expr)
Similar to `!procedure' and `!procedure*' for integers.
> (!test expr)
> (!test expr result [equal?])
The first form checks that the given expression evaluates to a
non-#f value in the submission context, throwing an error
otherwise. The second form compares the result of evaluation,
requiring it to be equal to `result' (optionally specifying an
equality procedure).
> (!all-covered)
When coverage information is enabled (see `:coverage?' above), this
procedure checks the collected coverage information and throws an
error with source information if some code is left uncovered. The
collected information includes only execution coverage by submission
code, excluding additional checker tests.
Reference in New Issue View Git Blame Copy Permalink