Proposal for the Committee on Meetings:

  A Judging Process for Contributed Papers

In this proposal, I describe a simple process that the Computing Section has discussed for improving the Joint Statistical Meetings. My reasons for making this suggestion are as follows:

  1. By introducing some form of judging, good papers can be better identified at the JSM.

    As any attendee of the JSM knows, it can be quite hard to judge anything about a talk from the paragraph-long abstract in the conference materials. This is a serious problem for members of the Program Committee who typically have to plan the program with only the short abstracts in hand. The process outlined below would provide Committee members with a more complete picture of the work to be presented.

  2. Through the judging process, we can help improve the overall quality of contributed papers and presentations at the JSM.

    The papers that pass the judging procedure will receive more prominent positioning in the JSM materials (both at the meeting and in the proceedings). This gives us an opportunity to raise expectations about presentation quality. In addition, the proceedings will improve because the authors will have had feedback from judges.

    The following bullet items describe the judging process. The basic ingredients appear to be well within the jurisdiction of the Computing Section's Program Chair.

    • The Computing Section announces to its membership that it is going to offer a VOLUNTARY extra step to those submitting papers to our section.

    • From those that agree to participate, we request an extended abstract that is at most one page in length, not including a reference list. This abstract is to be submitted in either PDF (Adobe's Portable Document Format) or in some form that can be directly transformed into PDF (examples being Word or PostScript).

    • The program chair assembles a group of judges that consists mainly of authors of papers that have previously passed this process. The authors of those papers that are judged favorably in year T agree to serve as judges for papers submitted in year T+1. (The authors in year T will be told of this obligation prior to submitting their paper for judging.) Of course, the program chair is free to add members to this group.

    • All extended abstracts are read by at least five judges and rated. This process requires some standardization, but there are numerous precedents for it in other disciplines.

    • Those papers receiving sufficiently high scores are culled into special topic sessions that are labeled

      COMPUTING SECTION REFEREED PAPERS: TOPIC X

      where TOPIC X is whatever the Program Chair thinks applies to the papers in the session. The refereed status will also be indicated in the proceedings and can be added to future references to the paper.

    • The extended abstracts will be translated to PDF (where necessary) and combined into a single extended-abstract program, available to the Computing Section's members (through our Web Site www.statcomputing.org ).

    • Those papers that are not among the highest rated retain regular contributed status and can opt for a regular contributed talk or a technical exhibition.

In socializing this idea with the Executive Committee of the Computing Section, there was some concern over the burden of the judging process. Personally, I see this process as a way to encourage new researchers to become more involved in the JSM and the ASA in general.