Grading
In the interest of expediency and flexibility
we have decided to perform the grading
in a separate process. Specifically,
we will use some sort of a script
(e.g. Perl) to create a grade for the lab.
The benefits of this approach
are
- we can ignore the details initially
and have a single method in the
general code to display the grade,
should that be required.
- instructors can change the grading scheme of
a lab by modifying or replacing the script.
- we need no mechanism for computing the grade
since we implement the script approach using
a mechanism already present in all the labs.
(See below).
The Scripts
Input to The Scripts
We want to make all the information available
to an instructor to perform the grading.
S/he might want to emphasize different aspects
or even simply monitor the areas the students don't seem to understand
or find overly simple, etc.
This information is (perhaps cryptically) contained in the
session information that is written out each time
the student quits the lab without completing it.
We use this information for the grading
since we have to be able to
store the state anyway.
When the student chooses to submit the
lab for grading, we simply dump the state or session information
at this time, describing the entire lab.
If we want to report the grade to the student immediately
before they quit the lab, we can do this by calling
the script from within the application and getting the
return value.
Local & Remote Use
There are two types1 of use of the lab.
These are
- stand alone labs that students run from within their class
accounts.
- web accessed applets
One of the goals and reasons for using Java, other than portability,
is to allow the students and non-students also to complete these labs
at home or at least on machines other than those located in the
department.
So we intend to allow the labs be accessed as applets on an http
server.
In this setup, we have two different types of users.
Stand Alone Applet
| / \
| / \
Student Student Anonymous
/ \ / \ / \
Quit Session Quit Session Quit Session
The way this works in the code is that the application
invokes the methods
getGradeInfoConnection
getSessionInfoConnection
to obtain an OutputStream object
onto which the grading or session/state information,
respectively is to be written.
From there, the method dumpState() is called
which goes through each card and dumps the state of each card
in the navigation window.
Format of the Session Files
The format of the session file
for direct use in grading
is as follows.
Header
Contains student id including the class account
and digits of the SID.
Separator
Card Information
Each Card is introduced by the separator above
and the next line is the name of the card.
Objects
A simple way of doing this is
to dump field-value pairs as a properties file for each object
and a different separator indicating the end of the object.
Implementation of the Remote HTTP Interface
The application opens the connection to one of two sources.
In
The application writes the session information
to the connection.
The text written out is prepended with
the user's id, etc. and then followed
by the content for each card.
This connection is most likely to
an http CGI script which reads the input
and mails it to a single class filter account.
This processes the mail message and writes the
data into a class directory for the user.
This is mirrored on the http site
also as an independent operation done daily or hourly, etc.
When the user starts the application the next time, the application
restores the information.
So each description & skeleton must specify exactly
what is required to preserve state
(i.e. the minimal information).
Also, the programmer must arrange to
write the objects.
This is annoying as we end up re-inventing the
persistent objects. Perhaps we should look at some of the
serialization discussions. See USENIX 1996 Fall.
Problems with This Approach
- Security.
If a student wants to count his or her score
for this lab, they must identify themselves when they start.
To do so, the must provide some information that allows
them "account related" access associated with the
class account on the department's computer system.
They do this by filling in a field in an HTML form
that is used to invoke the lab or with a input dialog window
that appears when they attempt to store the session
or grade information.
It is possible that network traffic could be monitored
and someone other than this student get access
to that "account related" material.
Note that the user can choose to enter no account name
and so act as an anonymous user. In that case, no
state information is stored but a grade is potentially
returned if the user has completed the lab (or if the script
can handle partial completion).
- There is little that can be done about this to eliminate it
completely, other than installing a secure encrypted
server. We won't do this. Instead, we make the problem
less likely to happen at a simple level by not using
the publicly available class account name. Instead,
we use a more private version of this, such as
the class account name followed by the last k digits
of the student's social security number. This is the
identifier the student enters when prompted.
- Update of mirror.
There will be a period (of known and controllable length)
in which the student's information from the http server
and the class account might be out synchronization.
- This interval can be shortened
considerably, and we could even make the copying immediate
with some sort of script. In short, this is not important.
- The students can logon as anonymous and complete the lab
multiple times and then logon using the class account
and get a perfect score.
- This is not even an issue at all as they are learning.
- responsibility is moved to the script, etc for handling
the grading and thus that must take care
of the different usages (stand-alone and applet).
We could do this in the application and avoid this
difference.
- The current approach is advantage for customization
and also allows us to produce the labs more quickly.
Notes
We should be able to go in insert the grading information
directly should we ever need to.
1
The general framework hopefully will hide the details from
the specifics of the application, and
the variable Globals.FromMain
is used in the cases where the code is conditional
on how it was invoked.
Duncan Temple Lang<duncan@research.bell-labs.com>
Last modified: Fri Apr 18 13:33:31 PDT 1997