Grading

In the interest of expediency and flexibility we have decided to perform the grading in a separate process. Specifically, we will use some sort of a script (e.g. Perl) to create a grade for the lab. The benefits of this approach are

The Scripts

Input to The Scripts

We want to make all the information available to an instructor to perform the grading. S/he might want to emphasize different aspects or even simply monitor the areas the students don't seem to understand or find overly simple, etc. This information is (perhaps cryptically) contained in the session information that is written out each time the student quits the lab without completing it.

We use this information for the grading since we have to be able to store the state anyway. When the student chooses to submit the lab for grading, we simply dump the state or session information at this time, describing the entire lab.

If we want to report the grade to the student immediately before they quit the lab, we can do this by calling the script from within the application and getting the return value.

Local & Remote Use

There are two types1 of use of the lab. These are
  1. stand alone labs that students run from within their class accounts.
  2. web accessed applets
One of the goals and reasons for using Java, other than portability, is to allow the students and non-students also to complete these labs at home or at least on machines other than those located in the department. So we intend to allow the labs be accessed as applets on an http server. In this setup, we have two different types of users.
    Stand Alone                Applet
       |                    /         \    
       |                   /           \
     Student            Student      Anonymous
     /     \            /     \      /       \
    Quit   Session    Quit  Session Quit    Session
The way this works in the code is that the application invokes the methods getGradeInfoConnection getSessionInfoConnection to obtain an OutputStream object onto which the grading or session/state information, respectively is to be written. From there, the method dumpState() is called which goes through each card and dumps the state of each card in the navigation window.

Format of the Session Files

The format of the session file for direct use in grading is as follows.

Header

Contains student id including the class account and digits of the SID.

Separator

Card Information

Each Card is introduced by the separator above and the next line is the name of the card.
Objects
A simple way of doing this is to dump field-value pairs as a properties file for each object and a different separator indicating the end of the object.

Implementation of the Remote HTTP Interface

The application opens the connection to one of two sources. In The application writes the session information to the connection. The text written out is prepended with the user's id, etc. and then followed by the content for each card. This connection is most likely to an http CGI script which reads the input and mails it to a single class filter account. This processes the mail message and writes the data into a class directory for the user. This is mirrored on the http site also as an independent operation done daily or hourly, etc. When the user starts the application the next time, the application restores the information.

So each description & skeleton must specify exactly what is required to preserve state (i.e. the minimal information). Also, the programmer must arrange to write the objects. This is annoying as we end up re-inventing the persistent objects. Perhaps we should look at some of the serialization discussions. See USENIX 1996 Fall.

Problems with This Approach

Security. If a student wants to count his or her score for this lab, they must identify themselves when they start. To do so, the must provide some information that allows them "account related" access associated with the class account on the department's computer system. They do this by filling in a field in an HTML form that is used to invoke the lab or with a input dialog window that appears when they attempt to store the session or grade information. It is possible that network traffic could be monitored and someone other than this student get access to that "account related" material.

Note that the user can choose to enter no account name and so act as an anonymous user. In that case, no state information is stored but a grade is potentially returned if the user has completed the lab (or if the script can handle partial completion).

There is little that can be done about this to eliminate it completely, other than installing a secure encrypted server. We won't do this. Instead, we make the problem less likely to happen at a simple level by not using the publicly available class account name. Instead, we use a more private version of this, such as the class account name followed by the last k digits of the student's social security number. This is the identifier the student enters when prompted.
Update of mirror. There will be a period (of known and controllable length) in which the student's information from the http server and the class account might be out synchronization.
This interval can be shortened considerably, and we could even make the copying immediate with some sort of script. In short, this is not important.
The students can logon as anonymous and complete the lab multiple times and then logon using the class account and get a perfect score.
This is not even an issue at all as they are learning.
responsibility is moved to the script, etc for handling the grading and thus that must take care of the different usages (stand-alone and applet). We could do this in the application and avoid this difference.
The current approach is advantage for customization and also allows us to produce the labs more quickly.

Notes

We should be able to go in insert the grading information directly should we ever need to.
1 The general framework hopefully will hide the details from the specifics of the application, and the variable Globals.FromMain is used in the cases where the code is conditional on how it was invoked.
Duncan Temple Lang<duncan@research.bell-labs.com>
Last modified: Fri Apr 18 13:33:31 PDT 1997