An S to Database Inteface
Many of the projects we are involved with here in Bell Labs
involve data that are maintained in SQL-compliant databases.
Traditionally, tools for statistical analysis have required
that the data be initially presented to them in flat ASCII files
or some other processed version of the original data source.
We are attempting to expand our horizons and
allow direct and dynamic communication with
databases from S and perhaps other statistical tools.
The intent is to reduce the user's managerial responsiblities
and complexities by requiring only one version of the data
to be stored.
This becomes an important issue in a dynamic system
when the data collection process is continuous and asynchronous.
Also, in the spirit of our
distributed computing project,
the connection between the statistical tool and the database
allows for certain computations (e.g. subsetting)
to be performed in the database. This has the potential
to allow the statistical tools to work with larger data sets
than previously possible.
While this can also be achieved by having the return values
from the database arrive as a stream so that the receiver
does not hold the entire object at a single time. Instead,
the processing is performed element-wise.
The approach of performing
some computations in the remotely, however, is a simpler
approach requiring less modifications to existing algorithms.
Java and the JDBC package
There are at least two obvious ways to implement this
functionality for a language such as S.
The first is to use a C interface from within
S (dynamically loaded as a chapter) which provides the low-level SQL
functionality by communicating directly with one or more databases.
The second approach uses a separate process as an interface
between the S level tasks and the database(s).
We have choosen to investigate the second of these approaches.
The reasons for this include:
- JDBC provides a simple interface to a range of SQL databases
each of which would require minor context changes in a C
level inteface.
- The interface is
easily created and maintained as much of the code exists
and requires simple connections to be established.
Implementation
Our proposed implementation exploits the
general distributed
computing environment
and an extraordinarily general way to provide
access to existing code in both S and Java with little or no
additional code. Really!
The way we do this is to use each of the following:
- Java Class Reflectance
- This allows us to generate the IDL describing
existing Java classes (i.e. the JDBC classes),
identifying the different methods and attributes.
This can be inserted directly into the interface
repository and from there stubs and skeletons generated.
This is implemented by a single application that
is similar to the java2idl/Caffeine application
from Visigenic
- CORBA's Dynamic Skeleton Interface
- This allows us to provide a single generic
Java class which provides the functionality
of an arbitrary Java object to CORBA clients
with no additional code.
The class converts CORBA requests into method
invocations on the associated object
by identifying the Java method
and converting the arguments.
Java Reflectance is again used to determine the method
and construct the call.
Essentially we are using the generic class
as a primitive Java interpreter.
(Note that the same mechanism allows us to provide
remote access to S (and generally interpreted languages such
as XLisp-stat, etc.) functions and objects acting
as CORBA servers.)
- CORBA's Dynamic Invocation Interface
- This is used on the S side to convert calls to
functions in S on a remote database object
into corresponding CORBA remote method invocations.
(e.g.
S > plot(x,a,b,c)
expands to a remote call to
x.plot(a,b,c).)
This and interrogation of the available IDL classes in the
CORBA interface repository allows us to generate
S methods for the remote methods on CORBA object references.
These three components allow us to utilize general remote
mechanisms in Java and S via the CORBA infrastructure
and require us to create only the Java application
that creates the required remote objects/servers.
The basic idea is to interface to the simple but general
parts of JDBC and allow arbitrary string commands be sent
from a client to the Java inteface client (see figure above).
Duncan Temple Lang<duncan@research.bell-labs.com>
Last modified: Fri May 29 19:22:58 EDT 1998