An S to Database Inteface

Many of the projects we are involved with here in Bell Labs involve data that are maintained in SQL-compliant databases. Traditionally, tools for statistical analysis have required that the data be initially presented to it as a flat ASCII file or some other processed version of the primary data. We are attempting to allow direct communication with a database from S and perhaps other statistical tools. This reduces the user's managerial complexities as only one version of the data is needed. This becomes an important issue in a dynamic system when the data collection process is continual. Also, in the spirit of our distributed computing project, the connection between the statistical tool and the database allows for certain computations (e.g. subsetting) to be performed in the database. This has the potential to allow the statistical tools to work with larger data sets than previously possible. This can also be achieved by having the return values from the database arrive as a stream so that the receiver does not hold the entire object at a single time. Instead, the processing is performed element-wise.

Java and the JDBC package

There are at least two obvious ways to implement this functionality for a language such as S. The first is to use a C interface from within S (dynamically loaded as a chapter) which provides the low-level SQL functionality by communicating directly with one or more databases. The second approach uses a separate process as an interface between the S level tasks and the database(s). We have choosen to investigate the second of these approaches. The reasons for this include
  1. JDBC provides a simple interface to a range of SQL databases each of which would require minor context changes in a C level inteface.
  2. easily created and maintained as much of the code exists and requires simple connections to be made,
  3. forms part of an experiment with distributed computing with data.

Implementation

Yet to be done!

The basic idea is to interface to the simple but general parts of JDBC and allow arbitrary string commands be sent from a client to the Java inteface client (see figure above).


Duncan Temple Lang<duncan@research.bell-labs.com>
Last modified: Mon Dec 8 23:37:11 EST 1997