_Handin Server and Client_

The "handin-server" directory contains a server to be run by a course
instructor for accepting homework assignments and reporting on
submitted assignments.

The "handin-client" directory contains a client to be customized then
re-distributed to students in the course.  The customized client will
embed a particular hostname and port where the server is running, as
well as a server certificate.

With a customized client, students simply install a ".plt" file --- so
there's no futzing with configuration dialogs and certificates.  A
student can install any number of clients at once (assuming that the
clients are properly customized, as described below).

The result, on the student's side, is a "Handin" button in DrScheme's
toolbar.  Clicking the "Handin" button allows the student to type a
password and upload the current content of the definitions and
interactions window to the course instructor's server.  The "File"
menu is also extended with a "Manage..." menu item for managing a
handin account (i.e., changing the password and other information, or
creating a new account if the instructor configures the server to
allow new accounts).  Students can submit joint work by submitting
with a concatenation of usernames separated by a "+".

On the instructor's side, the handin server can be configured to check
the student's submission before accepting it.

The handin process uses SSL, so it is effectively as secure as the
server and each user's password.


Quick Start for a Test Drive:
=============================

 1.  Create a new directory.

 2.  Copy "server-cert.pem" from the "handin-client" collection
     to the new directory.
       NOTE: for real use, you need a new certificate.
       NOTE: see also "Where is the collection?", below

 3.  Copy "private-key.pem" from the "handin-server" collection
     to the new directory.
       NOTE: for real use, you need a new key.

 4.  Create a file "users.ss" with the following content:
       ((tester ("8fe4c11451281c094a6578e6ddbf5eed")))

 5.  Make a "test" subdirectory in your new directory.

 6.  Create a file "config.ss" with the following content:
       ((active-dirs ("test")))

 7.  In your new directory, run
        mred -mvqM handin-server

 8.  In the "handin-client" collection, edit "info.ss" and
     uncomment the line
        (define server:port "localhost:7979")
     and the lines that define `tools', `tool-names', and
     `tool-icons'.

 9. Run `setup-plt -l handin-client' (on Windows, the executable is
    "Setup PLT", and you don't have to supply the command line
    arguments).

 10. Start DrScheme, click "Handin" to run the client, submit with
     username "tester" and password "pw".

     The submitted file will be .../test/tester/handin.scm.

 11. Check the status of your submission by pointing a web browser at
        https://localhost:7980/servlets/status.ss
     Note the "s" in "https". Use the "tester" username and "pw"
     password, as before.

  -------------------------------------------------------------------
 | "Where is the collection?"                                        |
 |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -|
 | If you obtained the server and client by installing a .plt file,  |
 | then the "handin-server" and "handin-client" directories          |
 | might be in your PLT addon space.  Start MzScheme, and enter      |
 |         (collection-path "handin-server")                         |
 |         (collection-path "handin-client")                         |
 | to find out where these collections are.                          |
  -------------------------------------------------------------------


Client Customization:
=====================

  1. Rename (or make a copy of) the "handin-client" collection
     directory.  The new name should describe your class uniquely.
     For example, "uu-cpsc2010" is a good name for CPSC 2010 at the
     University of Utah.

  2. Edit the first three definitions of "info.ss" in your renamed
     client collection:

       * For `name', choose a name for the handin tool as it will
         appear in DrScheme's interface (e.g., the "XXX" for the
         "Manage XXX Handin Account..."  menu item).  Again, make the
         name specific to the course, in case a student installs
         multiple handin tools.  Do not use "Handin" as the last part
         of the name, since "Handin" is always added for button and
         menu names.

       * For `server:port', uncomment the line, and use the hostname
         and port where the server will be running to accept handin
         submissions.

      Optionally uncomment and edit the next two definitions,
      `web-menu-name' and `web-address', to add an item to the "Help"
      menu that opens a (course-specific) web page.

   3. Replace "icon.png" in your renamed directory with a new 32x32
      icon.  This icon is displayed on startup with DrScheme's splash
      screen, and it is included at half size on the "Handin" button.
      Again, choose a distinct icon for the benefit of students who
      install multiple handin tools.  A school logo is typically
      useful, as it provides a recognizably local visual cue.

   4. Replace "server-cert.pem" in your renamed directory with a
      server certificate.  The file "server-cert.pem" in
      "handin-client" collection is ok for testing, but the point of
      this certificate is to make handins secure, so you should
      generate a new (self-certifying) certificate and keep its key
      private.  (See server setup, below.)

   5. Run
           mzc --collection-plt <name>.plt <name>
      where <name> is the name that you chose for your directory
      (i.e., whatever you changed "handin-client" to).

   6. Distribute <name>.plt to students for installation into their
      copies of DrScheme.  The students need not have access to the
      DrScheme installation directory; the tool will be installed on
      the filesystem in the student's personal space.  If you want to
      install it once on a shared installation, use setup-plt with the
      --all-users flag.


Server Setup:
=============

You must prepare a special directory to host the handin server.  To
run the server, you should either be in this directory, or you should
set the "PLT_HANDINSERVER_DIR" environment variable.

This directory contains the following files and sub-directories:

  * "server-cert.pem" --- the server's certificate.  To create a
    certificate and key with openssl:

      openssl req -new -nodes -x509 -days 365 -out server-cert.pem
                  -keyout private-key.pem

  * "private-key.pem" --- the private key to go with
    "server-cert.pem".  Whereas "server-cert.pem" gets distributed to
    students with the handin client, "private-key.pem" is kept
    private.

  * "config.ss" --- configuration options.  The file format is

         ((<key> <val>) ...)

    The following keys can be used (without the preceding quote):

      'active-dirs : a list of directories that are active
         submissions, relative to the current directory or absolute;
         the last path element for each of these (and 'inactive-dirs
         below) should be unique, and is used to identify the
         submission (for example, in the client's submission dialog
         and in the status servlet)

      'inactive-dirs : a list of inactive submission directories (see
         above for details)

      'port-number : the port for the main handin server; the default
         is 7979

      'https-port-number : the port for the handin-status HTTPS
         server; the default is one more than the main server's port
         (so the transitive default is 7980)

      'session-timeout : number of seconds before the session
         times-out -- the client is given this many seconds for the
         login stage and then starts again so the same number of
         seconds is given for the submit-validation process; the
         default is 300

      'session-memory-limit : maximum size in bytes of memory allowed
         for per-session computation, if per-session limits are
         supported (i.e., when using MrEd3m and MzScheme3m with memory
         accounting); the default is 40000000

      'default-file-name : the default filename that will be saved
        with the submission contents.  The default is "handin.scm".

      'max-upload : maximum size in bytes of an acceptable submission;
         the default is 500000

      'max-upload-keep : maximum index of submissions to keep; the
         most recent submission is "handin.scm" (by default), the next
         oldest is in "BACKUP-0/handin.scm", next oldest is
         "BACKUP-1/handin.scm", etc.; the default is 9

      'user-regexp : a regular expression that is used to validate
         usernames; alternatively, this can be `#f' meaning no
         restriction, or a list of permitted strings.  Young students
         often choose exotic usernames that are impossible to
         remember, and forget capitalization, so the default is fairly
         strict: #rx"^[a-z][a-z0-9]+$"; a "+" is always disallowed in
         a username, since it is used in a submission username to
         specify joint work

      'user-desc : a plain-words description of the acceptable
         username format (according to user-regexp above); `#f' stands
         for no description; the default is "alphanumeric string"
         which matches the default user-regexp

      'username-case-sensitive : a boolean; when `#f', usernames are
         case-folded for all purposes; defaults to `#f' (note that you
         should not set this to `#t' on Windows or when using other
         case-insensitive filesystems, since usernames are used as
         directory names)

      'allow-new-users : a boolean indicating whether to allow
         new-user requests from a client tool; the default is `#f'

      'allow-change-info : a boolean indicating whether to allow
         changing user information from a client tool (changing
         passwords is always possible); the default is `#f'

      'master-password : a string for an MD5 hash for a password that
         allows login as any user; the default is `#f', which disables
         the password

      'log-output : a boolean that controls whether the handin server
         log is written on the standard output; defaults to `#t'

      'log-file : a path (relative to handin server directory or
         absolute) that specifies a filename for the handin server log
         (possibly combined with the 'log-output option), or `#f' for
         no log file; defaults to "log"

      'web-base-dir : if `#f' (the default), the built-in web server
         will use the "status-web-root" in this collection for its
         configuration; to have complete control over the built in
         server, you can copy and edit "status-web-root", and add this
         configuration entry with the name of your new copy (relative
         to the handin server directory, or absolute)

      'web-log-file : a path (relative to handin server directory or
         absolute) that specifies a filename for logging the internal
         HTTPS status web server; or `#f' (the default) to disable
         this log

      'extra-fields : a list that describes extra string fields of
         information for student records; each element in this list is
         a list of three values -- the name of the field, the regexp
         (or `#f', or a list of permitted string values), and a string
         describing of acceptable strings.  The default is

           '(("Full Name" #f #f)
             ("ID#" #f #f)
             ("Email" #rx"^[^@<>\"`',]+@[a-zA-Z0-9_.-]+[.][a-zA-Z]+$"
              "a valid email address"))

         You can set this to a list of fields that you are interested
         in keeping, for example:

           '(("Full Name"
              #rx"^[A-Z][a-zA-Z]+(?: [A-Z][a-zA-Z]+)+$"
              "full name, no punctuations, properly capitalized")
             ("Utah ID Number"
              #rx"^[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]$"
              "Utah ID Number with exactly nine digits")
             ("Email"
              #rx"^[^@<>\"`',]+@cs\\.utah\\.edu$"
              "A Utah CS email address"))

         The order of these fields will be used both on the client GUI
         side and in the "users.ss" file (see below).

         The second item in a field description can also be the symbol
         '-, which marks this field as one that is hidden from the
         user interface: students will not see it and will not be able
         to provide or modify it; when a new student creates an
         account, such fields will be left empty.  This is useful for
         adding information that you have on students from another
         source, for example, adding information from a course roster.
         You should manually edit the "users.ss" file and fill in such
         information.  (The third element for such descriptors is
         ignored.)

      'hook-file : a path (relative to handin server directory or
         absolute) that specifies a filename that contains a `hook'
         module.  This is useful as a general device for customizing
         the server through Scheme code.  The file is expected to
         contain a module that provides a `hook' function, which
         should be receiving three arguments:
         - a symbol that indicates the operation that is now taking
           place;
         - a datum that specifies the connection context (a number for
           handin connections, a `wN' symbol for servlet connections,
           and #f for other server operations);
         - an alist of information relevant to this operation.
         Currently, the hook is used in several places after an
         operation has completed.  The first argument will be one of
         these symbols: 'server-start, 'server-connect, 'user-create,
         'user-change, 'login, 'submission-received,
         'submission-committed, 'submission-retrieved, 'status-login,
         'status-file-get.  For example, here is a simple hook module
         that sends notification messages when users are created or
         their information has changed:

           (module hook mzscheme
             (provide hook)
             (require (lib "sendmail.ss" "net"))
             (define (hook what session alist)
               (when (memq what '(user-create user-change))
                 (send-mail-message
                  "course-staff@university.edu"
                  (format "[server] ~a (~a)" what session)
                  '("course-staff@university.edu") '() '()
                  (map (lambda (key+val)
                         (apply format "~a: ~s" key+val))
                       alist)))))

    Changes to "config.ss" are detected, the file will be re-read, and
    options are reloaded.  A few options are fixed at startup time:
    port numbers, log file specs, and the `web-base-dir' are as
    configured at startup.  All other options will change the behavior
    of the running server (but things like `username-case-sensitive?'
    it would be unwise to do so).  (For safety, options are not
    reloaded until the file parses correctly, but make sure that you
    don't save a copy that has inconsistent options: it is best to
    create a new configuration file and move it over the old one, or
    use an editor that does so and not save until the new contents is
    ready.)  This is most useful for closing & openning submission
    directories.

  * "users.ss" (created if not present if a user is added) --- keeps
    the list of user accounts, along with the associated password
    (actually the MD5 hash of the password), and extra string fields
    as specified by the 'extra-fields configuration entry (in the same
    order).  The file format is

      ((<username-sym> (<pw-md5-str> <extra-field> ...))
        ...)

    For example, the default 'extra-field setting will make this:

      ((<username-sym> (<pw-md5-str> <full-name> <id> <email>))
        ...)

    Username that begin with "solution" are special.  They are used by
    the HTTPS status server.  Independent of the 'user-regexp and
    'username-case-sensitive? configuration items, usernames are not
    allowed to contain characters that are illegal in Windows
    pathnames, they cannot end or begin in spaces or periods.

    If the 'allow-new-users configuration allows new users, the
    "users.ss" file can be updated by the server with new users.  It
    can always be updated by the server to change passwords.

    If you have access to a standard Unix password file (from
    "/etc/passwd" or "/etc/shadow"), then you can construct a
    "users.ss" file that will allow users to use their normal
    passwords.  To achieve this, use a list with 'unix as the first
    element and the system's encrypted password string as the second
    element.  Such passwords can be used, but when users change them,
    a plain md5 hash will be used.

    You can combine this with other fields from the password file to
    create your "users.ss", but make sure you have information that
    matches your 'extra-fields specification.  For example, given this
    system file:

      foo:wRzN1u5q2SqRD:1203:1203:L.E. Foo:/home/foo:/bin/tcsh
      bar:$1$dKlU0OkJ$t63TzKz:1205:1205:Bar Z. Lie:/home/bar:/bin/bash

    you can create this "users.ss" file:

      ((foo ((unix "wRzN1u5q2SqRD") "L.E. Foo" "?"))
       (bar ((unix "$1$dKlU0OkJ$t63TzKz") "Bar Z. Lie" "?")))

    which can be combined with this setting for 'extra-fields in your
    "config.ss":

      ...
      (extra-fields (("Full Name" #f #f)
                     ("TA" '("Alice" "Bob") "Your TA")))
      ...

    and you can tell your students to use their department username
    and password, and use the "Manage ..." dialog to properly set
    their TA name.

    Finally, a password value can be a list that begins with a
    'plaintext symbol, which will be used without encryption.  This
    may be useful for manually resetting a forgotten passwords.

  * "log" (or any other name that the 'log-file configuration option
    specifies (if any), created if not present, appended otherwise)
    --- records connections and actions, where each entry is of the
    form
        (id time-str msg-str)
        [<id>|<time>] <msg>
    where `<id>' is an integer representing the connection (numbered
    consecutively from 1 when the server starts), "-" for a message
    without a connection, and "wN" for a message from the status
    servlet.

  * Active and inactive assignment directories (which you can put in a
    nested directory for convenience, or specify a different absolute
    directory), as specified by the configuration file using the
    `active-dirs' and `inactive-dirs'.  A list of active assignment
    directories (the last path element in each specified path is used
    as a label) is sent to the client tool when a student clicks
    "Handin".  The assignment labels are ordered in the student's menu
    using `string<?', and the first assignment is the default
    selection.

    Within each assignment directory, the student id is used for a
    sub-directory name.  Within each student sub-directory are
    directories for handin attempts and successes.  If a directory
    "ATTEMPT" exists, it contains the most recent (unsuccessful or
    currently-in-submission) handin attempt.  Directories "SUCCESS-n"
    (where n counts from 0) contain successful handins; the lowest
    numbered such directory represents the latest handin.

    A cleanup process in the server copies successful submissions to
    the student directory -- one level up from the corresponding
    "SUCCESS-n" directory.  This is done only for files and
    directories that are newer in "SUCCESS-n" than in the submission
    root, other files and directories are left intact.  If external
    tools add new content to the student directory (eg, a "grade"
    file, as described below) it will stay there.  If the machine
    crashes or the server is stopped, the cleanup process might not
    finish.  When the server is started, it automatically runs the
    cleanup process for each student directory.

    Within a student directory, a "handin.scm" file (or some other
    name if the `default-file-name' option is set) contains the actual
    submission.  A `checker' procedure can change this default file
    name, and it can create additional files in an "ATTEMPT" directory
    (to be copied by the cleanup process); see below for more details
    on "checker.ss".

    For submissions from a normal DrScheme frame, a submission file
    contains a copy of the student's definitions and interactions
    windows.  The file is in a binary format (to support non-text
    code), and opening the file directly in DrScheme shows the
    definitions part.  To get both the definitions and interactions
    parts, the file can be parsed with `unpack-submission' from
    "utils.ss" (see below).

    To submit an assignment as a group, students use a concatenation
    of usernames separated by "+" and any number of spaces (e.g.,
    "user1+user2").  The same syntax ("user1+user2") is used for the
    directory for shared submissions, where the usernames are always
    sorted so that directory names are deterministic.  Multiple
    submissions for a particular user in different groups will be
    rejected.

    Inactive assignment directories are used by the the HTTPS status
    web server.

  * "<active-assignment>/checker.ss" (optional) --- a module that
    exports a `checker' function.  This function receives two
    arguments: a username list and a submission as a byte string.
    (See also `unpack-submission', etc. from "util.ss", below.)  To
    reject the submission, the `checker' function can raise an
    exception; the exception message will be relayed back to the
    student.  The module is loaded when the current directory is the
    main server directory, so it can read files from there (but note
    that to read values from "config.ss" it is better to use
    `get-conf').  Also, the module will be reloaded if the checker
    file is modified -- no need to restart the server, but make sure
    that you do not save a broken checker (ie, do not save in
    mid-edit).

    The first argument is a list of usernames with at least one
    username, and more than one if this is a joint submission (where
    the submission username was a concatenation of usernames separated
    by "+").

    The `checker' function is called with the current directory as
    "<active-assignment>/<username(s)>/ATTEMPT", and the submission is
    saved in the file "handin", and the timeout clock is reset to the
    value of the 'session-timeout configuration.  The checker function
    can change "handin", and it can create additional files in this
    directory.  (Extra files in the current directory will be
    preserved as it is later renamed to "SUCCESS-0", and copied to the
    submission's root ("<active-assignment>/<username(s)>/"), etc.)
    To hide generated files from the HTTPS status web server
    interface, put the files in a subdirectory, it is preserved but
    hidden from the status interface.

    The checker should return a string, such as "handin.scm", to use
    in naming the submission file, or `#f' to indicate that he file
    should be deleted (eg, when the checker alrady created the
    submission file(s) in a different place).

    Alternatively, the module can bind `checker' to a list of three
    procedures: a pre-checker, a checker, and a post-checker.  All
    three are applied in exactly the same way as the checker (same
    arguments, and always within the submission directory), except
    that:
    - If there was an error during the pre-checker, and the submission
      directory does not have a "SUCCESS-*" directory, then the whole
      submission directory is removed.  This is useful for checking
      that the user/s are valid -- if you allow a submission only when
      `users' is '("foo" "bar"), and "foo" tries to submit alone, then
      the submission directory for "foo" should be removed to allow a
      proper submission later.  Note that the timeout clock is reset
      only once, before the pre-checker is used.
    - The post-checker is used at the end of the process, after the
      "ATTEMPT" directory was renamed to "SUCCESS-0".  At this stage,
      the submission is considered successful, so this function should
      avoid throwing an exception (it can, but the submission will
      still be in place).  This is useful for things like notifying
      the user of the successful submission (see `message' below), or
      sending a `receipt' email.
    To specify only pre/post-checker, use `#f' for the one you want to
    omit.

  * "<[in]active-assignment>/<user(s)>/<filename>" (if submitted) ---
    the most recent submission for <[in]active-assignment> by
    <user(s)> where <filename> was returned by the checker (or the
    value of the `default-file-name' configuration option if there's
    no checker).  If the submission is from multiple users, then
    "<user(s)>" is actually "<user1>+<user2>" etc.  Also, if the
    cleanup process was interrupted (by a machine failure, etc.), the
    submission may actually be in "SUCCESS-n" as described above, but
    will move up when the server performs a cleanup (or when
    restarted).

  * "<[in]active-assignment>/<user(s)>/grade" (optional) ---
    <user(s)>'s grade for <[in]active-assignment>, to be reported by
    the HTTPS status web server

  * "<[in]active-assignment>/solution*" --- the solution to the
    assignment, made available by the status server to any user who
    logs in.  The solution can be either a file or a directory with a
    name that begins with "solution".  In the first case, the status
    web server will have a "Solution" link to the file, and in the
    second case, all files in the "solution*" directory will be listed
    and accessible.

The server can be run within either MzScheme or MrEd, but "utils.ss"
requires MrEd (which means that `checker' modules will likely require
the server to run under MrEd).  It is best to use MrEd3m so memory
accounting is possible and the server will be protected from memory
related crashes.

The server currently provides no mechanism for a graceful shutdown,
but terminating the server is no worse than a network outage.  (In
particular, no data should be lost.)  The server reloads the
configuration file, checker modules etc, so there should not be any
need to restart it for reconfigurations.

The client and server are designed to be robust against network
problems and timeouts.  The client-side tool always provides a
"cancel" button for any network transaction.  For handins, "cancel" is
guaranteed to work up to the point that the client sends a "commit"
command; this command is sent only after the server is ready to record
the submission (having run it through the checker, if any), but before
renaming "ATTEMPT".  Also, the server responds to a commit with "ok"
only after it has written the file.  Thus, when the client-side tool
reports that the handin was successful, the report is reliable.
Meanwhile, the tool can also report successful cancels most of the
time.  In the (normally brief) time between a commit and an "ok"
response, the tool gives the student a suitable warning that the
cancel is unreliable.

To minimize human error, the number of active assignments should be
limited to one whenever possible.  When multiple assignments are
active, design a checker to help ensure that the student has selected
the correct assignment in the handin dialog.

A student can download his/her own submissions through a web server
that runs concurrently with the handin server.  The starting URL is

  https://SERVER:PORT/servlets/status.ss

to obtain a list of all assignments, or

  https://SERVER:PORT/servlets/status.ss?handin=ASSIGNMENT

to start with a specific assignment (named ASSIGNMENT).  The default
PORT is 7980.


Checker Utilities:
==================

The checker utilities are provided to make writing checker functions.
They are provided in a few layers, each layer provides new
functionality in addition to the lower one.  These modules are (in
order):

* sandbox.ss -- basic sandbox evaluation utilities, can be used
  independently from the handin-server.

* utils.ss -- additional utilities for dealing with handin
  submissions, as well as a few helpers for testing code.

* checker.ss -- this layer automates the task of creating a checker
  function (in "<active-assignment>/checker.ss" modules) to cope with
  common submission situations.

The following sections describe each of these modules.


_sandbox.ss_
------------

The main function that is implemented in this module is
`make-evaluator'.  Most of the functionality that is provided is used
by this function.

> mred?
  A boolean that is bound to `#t' if we're currently running in MrEd,
  `#f' if in plain MzScheme.  The idea is that you can use this module
  both from MzScheme or, if needed, from MrEd.  (Higher levels
  ("utils.ss" and "checker.ss"), need to be used with MrEd.)

> coverage-enabled
  A boolean parameter that controls whether coverage testing is
  enabled in `make-evaluator'-created functions.  If it set to true,
  the "handin-server/private/coverage.ss" module will be used to
  detect uncovered expressions.  This information is collected after
  the input port has been evaluated, so it is not affected by testing
  code that is not part of the submission.  To retrieve the collected
  information, apply the evaluation function on the special
  `get-uncovered-expressions' value below.  The resulting value is a
  list of uncovered expressions, with at most one per position+span
  (which means that the contents may be unreliable, but the position
  is).  The default is `#f'.

> (get-uncovered-expressions evaluator)
  When this is used with an evaluator that was created with
  `make-evaluator', it will return a list of uncovered syntax
  objects.  (It can also be provided as an argument to the evaluator,
  with the same result.)

> namespace-specs
  A parameter that holds a list of values that specify how to create a
  namespace for evaluation in `make-evaluator'.  The first item in the
  list is a thunk that creates the namespace, and the rest are require
  specs for modules that are to be attached to the created namespace.
  The default is `make-namespace' and `(lib "posn.ss" "lang")' if
  running in MzScheme, or `make-namespace-with-mred' and
  `(lib "cache-image-snip.ss" "mrlib")' as well as the posn library.
  (The module specs are needed for sharing module instantiations, for
  example, without the above, posn values in testing code will be a
  different type from posn values in tested code.)

> sandbox-reader
  A parameter that holds a function that reads all expressions from
  the current-input-port.  It is used to read the submission source.
  It must return a list of syntax objects, and it must use the symbol
  `program' as the input source (that is, something like a loop that
  consumes the input using `(read-syntax 'program)').  The default
  reader is using a plain `read-syntax' -- it does so while setting
  `read-case-sensitive' to `#t', and `read-decimal-as-inexact' to `#f'
  (both are sensible choices for testing code).

> sandbox-override-collection-paths
  A parameter that holds a list of collection directories.  A
  submission evaluator that is created by `make-evaluator' will put
  these directories (ones tat actually exist) in front of the
  collections in `current-library-collection-paths' -- so you can put
  collection overrides there.  The default is an `overridden-collects'
  directory in the handin-server collection, which comes with a few
  common overrides for teachpacks that use the GUI.

> sandbox-security-guard
  A parameter that holds a security guard that is used by all
  evaluations that happen in a `make-evaluator' function.  The default
  value is a security guard that forbids all I/O, except for things in
  `sandbox-path-permissions' (see below).

> sandbox-path-permissions
  This parameter configures the behavior of the default sandbox
  security guard by listing path and access modes that are allowed.
  The contents of this parameter is a list of specs, each one is a
  list of an access mode (a symbol) and a path spec that is granted
  this access.  The access mode symbol is one of: 'execute, 'write,
  'read, or 'exists, where each of these implies that modes that
  follow are also permitted (eg, 'read allows reading or checking for
  existence).  The path spec is either a path as a byte string (must
  be resolved and simplified) that will match exactly that path, or a
  byte-regexp that applies for all matching paths.  The default value
  is a list of 'read permissions for the library collection paths.
  Note that when an evaluator is created by `make-evaluator', the list
  is augmented with permissions for accessing non-`lib' teachpack
  requires and language module.

> sandbox-input
  A parameter that specifies the input for evaluations that happen in
  a `make-evaluator' function.  It defaults to `#f', which makes such
  functions work in a context where no input is available.  It can be
  set to:
  * an input port, which will be used as is;
  * a string or a byte string that will be used as the complete input;
  * a path that names a file holding the input.

> sandbox-output
  A parameter that specifies the output for evaluations that happen in
  a `make-evaluator' function.  It defaults to `#f', which simply
  discards all such output.  It can also be set to:
  * an output port, which will be used as is;
  * the symbol 'bytes, which will make `get-output' (see below) return
    the complete output as a byte string;
  * the symbol 'string, similar to the above, but uses a string;
  * the symbol 'pipe, which will make it use a pipe for output, and
    `get-output' returns the input end of the pipe.
  (Note that error output is *not* redirected.)

> (get-output evaluator)
  When this is used with an evaluator that was created with
  `make-evaluator', it will return the output of the evaluator.  (It
  can also be provided as an argument to the evaluator, with the same
  result.)  The result depends on the value of the `sandbox-output'
  parameter at the time the evaluator was created: if it was `#f' then
  `get-output' will return `#f', if it was the symbol `pipe' then
  `get-output' returns an input port that is being fed by the pipe,
  and if it was the symbol `bytes' or `string' then `get-output'
  returns the accumulated output and resets the evaluator's output to
  a new output string or byte string (so each call returns a piece of
  the evaluator's output).

> (make-evaluator language teachpack-paths input-program)
  This is the main entry point for the sandbox module.

  This function Creates an evaluator function for evaluating
  expressions in the designated `language', after loading teachpacks
  that are specified in `teachpack-paths', and after evaluating the
  code in the `input-program'.

  The `input-program' holds the input program in the same way as the
  `sandbox-input' parameter (but it cannot be `#f').  The contents of
  this input is read using the `sandbox-reader', with line-counting
  enabled.

  The `language' can be:
  * a symbol indicating a built-in language (currently, only
    'mzscheme), or a teaching language -- one of 'beginner,
    'beginner-abbr, 'intermediate, 'intermediate-lambda, or 'advanced.
  * a list that begins with a 'lib, 'file, or 'planet symbol, which
    stands for the language defined by this (quoted) module
    specification, or a string specifying a relative module filename
    directly.
  * a list that begins with a 'begin symbol means that the code will
    not be evaluated in a module context at all, it will simply be
    evaluated in a new namespace, after evaluating this list.

  The `teachpack-paths' list specifies additional code to load, can be
  one of:
  * paths to teachpacks to load into the module.
  * a list that begins with a 'begin symbol is arbitrary code that is
    prefixed into the submitted program.

  The actual evaluation of expressions (both the program and later
  evaluations) happens under the `sandbox-security-guard'
  restrictions, and if MrEd is used -- in a newly created eventspace.
  See also `with-limits' below for adding resource limits, and
  `get-uncovered-expressions' above for enforcing test coverage.

> (call-with-limits sec mb thunk)
  This function executes the given thunk with memory and time
  restrictions: if execution consumes more than `mb' megabytes or more
  that `sec' seconds, then the computation is aborted and an error is
  thrown.  Otherwise the result of the thunk is returned (a value,
  multiple values, or raise an exception).  Each of the two limits can
  be `#f' to disable it.

  (Note: memory limit requires running in a 3m executable.)

> (with-limits sec mb body ...)
  A macro version of the above.


_utils.ss_
----------

> (get-conf key)
  Returns a value from the configuration file (useful for reading
  things like field names etc)

> (unpack-submission bytes)
  Returns two text% objects corresponding to the submitted definitions
  and interactions windows.

> (make-evaluator/submission language teachpack-paths bytes)
  Like `make-evaluator', but the definitions content is supplied as a
  submission byte string.  The byte string is opened for reading, with
  line-counting enabled.

> (call-with-evaluator language teachpack-paths input-program proc)
  Calls `proc' with an evaluator for the given language, teachpack
  paths, and initial definition content as supplied by input-program
  (see `make-evaluator').  It also sets the current error-value print
  handler to print values in a way suitable for `lang', it initializes
  `set-run-status' with "executing your code", and it catches all
  exceptions to re-raise them in a form suitable as a submission
  error.

> (call-with-evaluator/submission language teachpack-paths bytes proc)
  Like `call-with-evaluator', but the definitions content is supplied
  as a submission string.  The byte string is opened for reading, with
  line-counting enabled.

> (evaluate-all source input-port eval)
  Like `load' on an input port.

> (evaluate-submission bytes eval)
  Like `load' on a submission byte string.

> (check-proc eval expect-v compare-proc proc-name arg ...)
  Calls the function named `proc-name' using the evaluator `eval',
  giving it the (unquoted) arguments `arg'...  Let `result-v' be the
  result of the call; unless `(compare-proc result-v expect-v)' is
  true, an exception is raised.

  Every exception or result mismatch during the call to `check-proc'
  phrased suitably for the handin client.

> (check-defined eval name)
  Checks whether `name' is defined in the evaluator `eval', and raises
  an error if not (suitably phrased for the handin client).  If it is
  defined as non-syntax, its value is returned.  Warning: in the
  beginner language level, procedure definitions are bound as syntax.

> (look-for-tests text name n)
  Inspects the given text% object to determine whether it contains at
  least `n' tests for the function `name'.  The tests must be
  top-level expressions.

> (user-construct eval name arg ...)
  Like `check-proc', but with no result checking.  This function is
  often useful for calling a student-defined constructor.

> test-history-enabled
  Parameter that controls how run-time errors are reported to the
  handin client.  If the parameter's value is true, then the complete
  sequence of tested expressions is reported to the handin client for
  any test failure.  Set this parameter to true when testing programs
  that use state.

> (message string [styles])
  If given only a string, this string will be shown on the client's
  submission dialog; if `styles' is also given, it can be the symbol
  'final, which will be used as the text on the handin dialog after a
  successful submission instead of "Handin successful." (useful for
  submissions that were saved, but had problems); finally, `styles'
  can be used as a list of styles for a `message-box' dialog on the
  client side, and the resulting value is returned as the result of
  `message'.  You can use that to send warnings to the student and
  wait for confirmation.

> (set-run-status string-or-#f)
  Registers information about the current actions of the checker, in
  case the session is terminated due to excessive memory consumption.
  For example, a checker might set the status to indicate which
  instructor-supplied test was being executed when the session ran out
  of memory.  This status is only used when per-session memory limits
  are supported (i.e., under MrEd3m or MzScheme3m with memory
  accounting), but in both cases, a string value will also be passed
  on to `message' above.

> (current-value-printer proc)
  A parameter that controls how values are printed, a procedure that
  expects a Scheme value and returns a string representation for it.
  The default value printer uses pretty-print, with DrScheme-like
  settings.

> (reraise-exn-as-submission-problem thunk)
  Calls thunk in a context that catches exceptions and re-raises them
  in a form suitable as a submission error.

> (log-line fmt args ...)
  Produces a line in the server log file, using the given format
  string and arguments.  All this actually does, is arrange to print
  the line fast (to avoid mixing lines from different threads) to the
  error port, and flush it.  (The log port will prefix all lines with
  a time stamp and connection identifier.)

> (timeout-control msg)
  Control the timeout for this session.  The timeout is initialized by
  the value of the 'session-timeout configuration entry, and the
  checker can use this procedure to further control it: if msg is
  'reset the timeout is reset to 'session-timeout seconds; if msg is a
  number the timeout will be set to that many seconds in the future.
  The timeout can be completely disabled by `(timeout-control #f)'.
  (Note that before the checker is used (after the pre-checker, if
  specified), the timer will be reset to the 'session-timeout value.)


_checker.ss_
------------

The "checker.ss" module provides a higher-level of utilities, helpful
in implementing `checker' functions that are intended for a more
automated system.  This module is a language module -- a typical
checker that uses it looks like this:

   (module checker (lib "checker.ss" "handin-server")
     (check: :language  'intermediate
             :users     pairs-or-singles-with-warning
             :coverage? #t
       (!procedure Fahrenheit->Celsius 1)
       (!test (Fahrenheit->Celsius  32)   0)
       (!test (Fahrenheit->Celsius 212) 100)
       (!test (Fahrenheit->Celsius  -4) -20)
       ...))

> (check: :key val ... body ...)
  Construct a checker procedure.

The `check:' macro will construct an appropriate checker function,
using keywords for features that you want, the body of the checker can
contain arbitrary code, using all utilities from "utils.ss" (see
above), as well as additional ones (see below).

Keywords for configuring `check:':

* :users -- specification of users that are acceptable for
  submission.  Can be either a list of user lists, each representing a
  known team, or procedure which will accept a list of users and throw
  an exception if they are unacceptable.  The default is to accept
  only single-user submissions.  `pairs-or-singles-with-warning' is a
  useful value for pair submission where the pairs are unknown (see
  below).

* :eval? -- whether submissions should be evaluated.  Defaults to
  `#t'.  Note that if it is specified as `#f', then the checker body
  will not be able to run any tests on the code, unless it contains
  code that performs some evaluation (eg, using the facilities of
  "utils.ss").

* :language -- the language that is used for evaluating submissions,
  same as the `language' argument for `make-evaluator' (see above).
  There is no default for this, so it must be set or an error is
  raised.

* :teachpacks -- teachpacks for evaluating submissions, same as the
  `teachpacks' argument for `make-evaluator' (see above).  Defaults to
  null -- no teachpacks.

* :create-text? -- if true, then a textual version of the submission
  is saved as "text.scm" in a "grading" subdirectory (or any suffix
  that is specified by :output below, for example "hw.java" is
  converted into a textual "grading/text.java").  This is intended for
  printouts and grading, and is in a subdirectory so students will not
  see it on the status web server.  Defaults to `#t'.

* :untabify? -- if true, then tabs are converted to spaces, assuming a
  standard tab width of 8 places.  This is needed for a correct
  computation of line lengths, but note that DrScheme does not insert
  tabs in Scheme mode.  Defaults to `#t'.

* :textualize? -- if true, then all submissions are converted to text,
  trying to convert objects like comment boxes and test cases to some
  form of text.  Defaults to `#f', meaning that an exception is raised
  for submissions that are not all text.

* :maxwidth -- a number that specifies maximum line lengths for
  submissions (a helpful feature for reading student code).  Defaults
  to 79.  This feature can be disabled if set to `#f'.

* :output -- the name of the original handin file (unrelated to the
  text-converted files).  Defaults to "hw.scm".  (The suffix changes
  the defaults of `:markup-prefix' and `:prefix-re' below.)  Can be
  `#f' for removing the original file after processing.

* :multi-file -- by default, this is set to `#f', which means that
  only DrScheme is used to send submissions as usual.  See
  "Multiple-file submissions" below for setting up multi-file
  submissions.

* :names-checker -- used for multi-file submissions; see
  "Multiple-file submissions" below for details.

* :markup-prefix -- used as the prefix for :student-lines and
  :extra-lines below.  The default is ";;> " or "//> ", depending on
  the suffix of :output above.  (Note: if you change this, make sure
  to change :prefix-re too.)

* :prefix-re -- used to identify lines with markup (";>" or "//>"
  etc), so students cannot fool the system by writing marked-up code.
  The default is ";>" or "//>", depending on the suffix of :output
  above.

* :student-line -- when a submission is converted to text, it begins
  with lines describing the students that have submitted it; this is
  used to specify the format of these lines.  It is a string with
  holes that that `user-substs' (see below) fills out.  The default is
  "Student: {username} ({Full Name} <{Email}>)", which requires a
  "Full Name" and "Email" entries in the server's extra-fields
  configuration.  These lines are prefixed with ";;> " or the prefix
  specified by :makup-prefix above.

* :extra-lines -- a list of lines to add after the student lines, all
  with a ";;> " or :markup-prefix too.  Defaults to a single line:
  "Maximum points for this assignment: <+100>".  (Can use
  "{submission}" for the submission directory.)

* :user-error-message -- a string that is used to report an error that
  occurred during evaluation of the submitted code (not during
  additional tests).  It can be a plain string which will be used as
  the error message, or a string with single a "~a" (or "~e", "~s",
  "~v") that will be used as a format string with the actual error
  message.  The default is "Error in your code --\n~a".  Useful
  examples of these messages:

    "There is an error in your program, hit \"Run\" to debug"
    "There is an error in your program:\n----\n~a\n----\n
     Hit \"Run\" and debug your code."

  Alternatively, the value can be a procedure that will be invoked
  with the error message.  The procedure can do anything it wants, and
  if it does not raise an exception, then the checker will proceed as
  usual.  For example:

    (lambda (msg)
      (add-header-line! "Erroneous submission!")
      (add-header-line! (format "  --> ~a" msg))
      (message (string-append
                "You have an error in your program -- please hit"
                " \"Run\" and debug your code.\n"
                "Email the course staff if you think your code is"
                " fine.\n"
                "(The submission has been saved but marked as"
                " erroneous.)")
               '(ok))
      (message "Handin saved as erroneous." 'final))

  (Note that if you do this, then additional tests should be adjusted
  to not raise an exception too.)

* :value-printer -- if specified, this will be used for
  `current-value-printer' (see above).

* :coverage? -- collect coverage information when evaluating the
  submission.  This will cause an error if some input is not covered.
  This check happens after checker tests are run, but the information
  is collected and stored before, so checker tests do not change the
  result.  Also, you can use the `!all-covered' procedure in the
  checker before other tests, if you want that feedback earlier.

Within the body of `check:', `users' and `submission' will be bound to
the checker arguments -- a (sorted) list of usernames and the
submission as a byte string.  In addition to the functionality below,
you can use `((submission-eval) expr)' to evaluate expressions in the
submitted code context, and you can use `(with-submission-bindings (id
...) body ...)' to evaluate the body when `id's are bound to their
value from the submission code.

> (pre: body ...)
> (post: body ...)
  These two macros define a pre- and a post-checker.  In their body,
  `users' and `submission' are bound as in `check:', but there is
  nothing else special about these.  See the description of the
  `pre-checker' and `post-checker' values for what can be done with
  these, and note that the check for valid users is always first.  An
  example for a sophisticated `post:' block is below -- it will first
  disable timeouts for this session, then it will send a email with a
  submission receipt, with CC to the TA (assuming a single TA), and
  pop-up a message telling the student about it:

  (require (lib "sendmail.ss" "net"))
  (post:
    (define info
      (format "hw.scm: ~a ~a"
              (file-size "hw.scm")
              (file-or-directory-modify-seconds "hw.scm")))
    (timeout-control 'disable)
    (log-line "Sending a receipt: ~a" info)
    (send-mail-message
     "course-staff@university.edu"
     "Submission Receipt"
     (map (lambda (user) (user-substs user "{Full Name} <{Email}>"))
          users)
     (list (user-substs (car users) "{TA Name} <{TA Email}>"))
     null
     `("Your submission was received" ,info))
    (message (string-append
              "Your submission was successfully saved."
              "  You will get an email receipt within 30 minutes;"
              " if not, please contact the course staff.")
             '(ok)))

> submission-eval
  A parameter that holds an evaluation procedure for evaluating code
  in the submission context.

> (user-data user)
  Returns a user information given a username.  The returned
  information is a list of strings that corresponds to the configured
  `extra-fields' (see above).

> (user-substs user str)
  Given a username, this procedure will lookup the user's extra-fields
  information (see above) and then substitute field names in {braces}
  by the corresponding value.  An error will be signaled if a field
  name is missing.  Also, "{username}" will always be replaced by the
  username and "{submission}" by the current submission directory.

  This is used to process the `:student-line' value in the checker,
  but it is provided for additional uses.  See the above sample code
  for `post:' for using this procedure.

> (pairs-or-singles-with-warning users)
  This procedure is intended for use as the :users entry in a checker.
  It will do nothing if there are two users, and throw an error if
  there are more.  If there is a single user, then the user will be
  asked to verify a single submission -- if the students cancels, then
  an exception is raised so the submission directory is retracted.  If
  the student approves this, the question is not repeated (this is
  marked by creating a directory with a known name).  This is useful
  for cases where you want to allow free pair submissions -- students
  will often try to submit their work alone, and later on re-submit
  with a partner.

> (teams-in-file team-file)
  This procedure *returns* a procedure that can be used for the :users
  entry in a checker.  The team file (relative from the server's main
  directory) is expected to have user entries -- a sequence of
  s-expressions, each one a string or a list of strings.  The
  resulting procedure will allow submission only by teams that are
  specified in this file.  Furthermore, if this file is modified, the
  new contents will be used immediately, so there is no need to
  restart the server of you want to change student teams.  (But
  remember that if you change ("foo" "bar") to ("foo" "baz"), and
  there is already a "bar+foo" submission directory, then the system
  will not allow "foo" to submit with "bar".)

> (add-header-line! line)
  During the checker operation, this procedure can be used to add
  header lines to the text version of the submitted file.  It will not
  have an effect if `:create-text?' is false.

> (procedure/arity? proc arity)
  Returns `#t' if `proc' is a procedure that accepts `arity'
  arguments.

> (!defined id ...)
  A macro that checks that the given identifiers are defined in the
  (evaluated) submission, and throws an error otherwise.

> (!procedure id [arity])
> (!procedure* expr [arity])
  `!procedure' checks that `id' is defined, and is bound to a
  procedure; `!*procedure' is similar but omits the defined check,
  making it usable with any expression (which is evaluated in the
  submission context).

> (!integer id)
> (!integer* expr)
  Similar to `!procedure' and `!procedure*' for integers.

> (!test expr)
> (!test expr result [equal?])
  The first form checks that the given expression evaluates to a
  non-`#f' value in the submission context, throwing an error
  otherwise.  The second form compares the result of evaluation,
  requiring it to be equal to `result' (optionally specifying an
  equality procedure).  Note that the `result' and `equal?' forms are
  *not* evaluated in the submission context.

> (!all-covered)
  When coverage information is enabled (see `:coverage?' above), this
  procedure checks the collected coverage information and throws an
  error with source information if some code is left uncovered.  The
  collected information includes only execution coverage by submission
  code, excluding additional checker tests.  You do not have to call
  this explicitly -- it is called at the end of the process
  automatically when `:coverage?' is enabled.  It is made available so
  you can call it earlier (eg, before testing) to show clients a
  coverage error first.


Multiple-File Submissions:
==========================

By default, the system is set up for submissions of single a single
file, straight fom DrScheme using the handin-client.  There is some
support for multi-file submissions in "checker.ss" and in the
handin-client -- it is possible to submit multiple files, and have the
system generate a single file that is the concatenation of all
submission files (used only with text files).  To set up multi-file
submissions, do the following:

* Add a `:multi-file' keyword in `check:', and as a value, use the
  suffix that should be used for the single concatenated output file.

* You can also add a `:names-checker' keyword -- the value can be a
  regexp that all submitted files must follow (eg, ".*[.]scm$"), or a
  list of expected file names.  Alternatively, it can be a 1-argument
  procedure that will receive the (sorted) list of submitted files and
  can throw an error if some files are missing or some files are
  forbidden.

* In the "info.ss" file of the handin-client you need to set
  `enable-multifile-handin' to `#t', and adjust `selection-default' to
  patterns that are common to your course.  (It can be a single
  pattern, or a list of them.)

On the server side, each submission is saved in a file called "raw",
which contains all submitted files.  In the "grading" directory, you
will get a "text.<sfx>" file ("<sfx>" is the suffix that is used as a
value for `:multi-file') that contains all submitted files with clear
separators.  A possible confusion is that every submission is a
complete set of files that overwrites any existing submission -- but
students may think that the server accumulates incoming files.  To
avoid such confusion, when a submission arrives an there is already an
existing previous submission, the contents is compared, and if there
are files that existed in the old submission but not in the new ones,
the student will see a warning pop-up that allows aborting the
submission.

On the client side, students will have an additional file-menu entry
for submitting multiple files, which pops up a dialog that can be used
to submit multiple files.  In this dialog, students choose their
working directory, and the `selection-default' entry from the
"handin-client/info.ss" file specifies a few patterns that can be used
to automatically select files.  The dialog provides all handin-related
functionality that is available in DrScheme.  For further convenience,
it can be used as a standalone application: in the account management
dialog, the "Un/Install" tab has a button that will ask for a
directory where it will create an executable for the multi-file
submission utility -- the resulting executable can be used outside of
DrScheme (but PLT Scheme is still required, so it cannot be
uninstalled).


Auto-Updater:
=============

The handin-client has code that can be used for automatic updating of
clients.  This can be useful for courses where you distribute some
additional functionality (collections, teachpacks, language-levels
etc), and this functionality can change (or expected to change, for
example, distributing per-homework teachpacks).

To enable this, uncomment the relevant part of the "info.ss" file in
the client code, it has the following three keys: `enable-auto-update'
that turns this facility on, `version-filename' and `package-filename'
which are the expected file names of the version file and the .plt
file relative to the course web address (the value of the
`web-address' key).  Also, include in your client collection a
"version" file that contains a single number that is its version -- a
big integer that holds the time of this collection in a YYYYMMDDHHMM
format.

When students install the client, every time DrScheme starts, it will
automatically check the version from the web page (as specified by the
`web-address' and `version-filename' keys), and if that contains a
bigger number, it will offer the students to download and install the
new version.  So, every time you want to distribute a new version, you
build a new .plt file that contains a new version file, then copy
these version and .plt files to your web page, and students will be
notified automatically.  Note: to get this to work, you need to create
your .plt file using mzc's `--replace' flag, so it will be possible to
overwrite existing files.  (Also note that there is no way to delete
files when a new .plt is installed.)