Overview Talks Other Work

Distributed Computing With Data

The goal of one research project here in Bell Labs is to unite with other statistical software developers to provide an extended statistical computing environment in which users can avail of services from the different packages and applications from their favorite user interface (command line or suite of task specific GUIs). The intent is to allow the user focus on the statistical tasks and not manipulating the software and inputs. In short, users should be able refer to data in remote databases, apply statistical procedures available in different packages and plot the results in other specialized software - all as if the facilities were local to the language with which the user is familiar.

There are three different levels at which we might be able to communicate:

  1. process to process involving lower-level languages (C/C++ and Java)
  2. high level language to high level language - e.g. evaluating commands in one language from the "command line" of another. For example, in S we might evaluate the commands in Matlab to produce a plot. This is sufficient for high performance computing.
  3. direct invocation of methods for objects in high level languages. In this way, one can write code in one's favorite language and publish objects so that others can invoke methods on them supplying both their own local data or other remote objects. This is the goal which will allow users of different statistical environments to work as if they were one. In this way, the data and methods can be distributed across machines and languages and could be available across the web with little or no changes to the development or user interaction.
Another article on this topic.

It turns out that this same mechanism can provide us with the ability to offer High Performance Computing using clusters of networked machines from high level statistical languages. In this way, we should be able to efficiently perform lengthy calculations. Our efforts will provide means to (at least partially) automate the procedure of converting sequential algorithms written in S to parallel algorithms. Additionally, it will provide users with a rapid prototyping language for experimenting with different parallel approaches.

In order to attain this goal, much work is needed. But less than one might think as now other people are developing the technology on which we may provide our contributions. This site presents information regarding these technologies and encourages

Talks, Papers and Demonstrations

Joint Statistical Meetings 98 - PDF
This is the more complete version of another talk and was given at the '98 Joint Statistical meetings in Dallas. It describes our use of the dynamic facilities in CORBA to provide bindings for languages such as S, Xlisp-Stat. This consists of a single set of general routines providing transparent client and server access to CORBA. This also provides a high-level interface in Java.
``Statistics and the Internet'' Talk - PDF.
This is a talk given at the Statistics and the Internet conference in July. It outlines the procedure for providing simple client and server access to CORBA from interactive languages such as S and Xlisp-Stat and also Java. This approach employs the dynamic facilities in CORBA (DSI and DII).
Interface 98 Talk - PDF
A talk we gave in May 98 describing the architecture we will use (CORBA) for our distributed computing environment for data and the higher level tools we can develop with this.
CORBA and Other Distributed Resources
A small but growing collection of URLs, books, etc. on different aspects of distributed computing including the different systems that we are aware of that might be used as the infrastructure for a distributed statistical computing environment.
Comparison of the different systems.
This describes our argument for using CORBA for a general system and lists the pros and cons of the other systems. In order to evaluate the utility of these different technologies, we suggest a relatively common example. This is described as a Distributed Statistical Example.
Experiences with different ORB implementations.
Some comments on our early experiences dealing with different CORBA implementations and developing code to support each. We are currently using the OmniBroker in our work and are awaiting Java bindings to be implemented for TAO and OmniORB.
CORBA Tutorial using the Wafer Visualization Demo.
Details of the code used to implement the Wafer yield monitoring example presented in the Interface98 talk. This is intended to be a interactive figure leading to the different code segments
Statistical Data Types
A suggestion for creating Common Statistical Data Classes in IDL to facilitate the data transfer between different applications.

Previous Work

The work extends/replaces the S/Java interface developed previously. This uses the S object ASCII format to communicate data and objects between clients. We are separating the content of the message from the communication mechanism to allow different scenarios to be easily integrated. Additionally, we allow other message types to be transmitted in some manner, agreed upon by the peers involved in the communication.
Duncan Temple Lang<duncan@research.bell-labs.com>
Last modified: Mon Aug 17 08:37:19 EDT 1998