Functions, Methods, and OOP for Statistical Computing

In looking at where computing for data analysis should be heading in the near future, we believe that some (related) points stand out. The Omegahat software contains a variety of components directed at new directions for statistical. Part of the software constitutes an environment for computing with data based on OOP ideas.

Another question that arises is: How should we get from here to there? In particular, how can we start from the S language as currently used via R and S-Plus, and successfully draw people towards the new software?

We are implementing and describing an approach to this. S language users are presented with a set of gradual steps, starting from "non-programming" use of the language:

  1. Using and writing S functions.
  2. Specializing functions to methods, chosen according to the class of the argument(s).
  3. Designing classes of objects, in terms of the data they contain and the inheritance relations between classes.
  4. Object-oriented programming in the S language, with methods associated with classes of objects and with object references.
  5. Inter-system interfaces between the S language and other software.
  6. Object-oriented programming outside S, using the Omegahat environment, CORBA, and/or other software.
The present file discusses how software to support these steps is currently being implemented, along with some possibilities for S in general, and for R in particular.

A draft table of contents and preface for a planned book suggests how the ideas will be presented to users. Our belief is that for users to find these new ideas acceptable, and for them to benefit from them, the steps need to be gradual, the concepts should be kept as consistent as possible, and the results of their programming should be good, reusable software.

We are also committed to making all of the software available in both S-Plus and R: in the current situation, it's a disservice to users to trap them into one or the other implementation, if there is any way to avoid doing so.

Here are some details coming out of this approach.

Object-Oriented Programming and S

The OOP model, as we present it, has two main features: methods are associated with objects (via their class and superclasses); and object references are provided that allow changes (mutation) of the object by non-local software.

We don't argue that the OOP model should replace the S semantics, but we do suggest situations where it is a preferred alternative. Associating methods with classes makes it easier to write an inter-related set of methods designed for a particular class of objects. Objects that are naturally set up to respond to diverse events fit more naturally into the OOP model. Components of a graphical user interface are obvious examples. More traditional objects, such as complex statistical models, can also benefit from an OOP perspective, because it allows these objects, also, to respond to events.

Even if we didn't want to implement the OOP model in S for its own merits, it plays an important role in implementing a good inter-system interface, both for Java and for CORBA. In many applications of inter-system interfaces, there is a strong argument for not copying data between the systems. Efficiency arguments can argue strongly for keeping the data where it belongs, but there are stronger arguments. If the structure of the data is not trivial, replicating the structure in the other system may be hard, or even not totally meaningful. If the data may be changed during the interaction between the systems, copying makes for well-known problems of synchronization.

The right way to provide data across systems, in our opinion, is for the data to reside where it makes most sense for the application, and to define the data by an interface (in the CORBA or Java sense) to the other system(s). CORBA almost requires this approach, but it works well for other systems as well. In particular, it's a natural way for S to provide data to other systems (e.g., to Java or to other S language systems, via CORBA).

OOP in S comes into the picture if the remote system is to be allowed to modify the S object. Modification will be provided via some methods in the interface; the implementation of these methods doesn't fit easily in the traditional S semantics because of the lack of object references. But once one implements OOP, this provides the natural semantics required (not surprisingly, since we set out to implement an OOP model similar to the one underlying CORBA or Java).

S4-Style Classes and Methods in R

There are some good arguments in favor of providing in R the essential features of classes and methods as defined in Programming with Data.

Overall, it seems both desirable and feasible to implement these features in R.

This probably does not mean that the R implementation can or should replicate everything in the corresponding chapters of the green book, certainly not at this stage. (Since S-Plus does not currently support all the S4 features documented in the book, the pressure for completeness isn't strong.) Some essential features that also seem feasible include the following.

  1. Programmers should be able to specify a method corresponding to an arbitrary pattern of classes for any subset of the formal arguments of any S function (excluding the "..." argument, but including the notion of matching "missing" to missing arguments).
  2. Programmers should be able to define a class of objects, in terms of the names and classes of the data contained in the object (the "slots") and also in terms of other classes that this class extends.
  3. The information defined by 1 and 2 should be available dynamically; that is, the programmer should be able to determine the current methods associated with a function, the current definition of a class, and the inheritance relations between classes.
We have a partially complete implementation of these features. See directory R/Classes/ in the Omegahat source tree. They are sufficient to support most of the utilities in the green book, although not including some of the optional features. (For example, object validity methods are not supported.)

The current implementation functions as an add-on, written entirely in the S language. This will likely change a little bit, at least in that some C code will be needed. One basic difficulty is the current lack of any "session-scope" storage facilities in R (analogous to the S "frame 0" database (Blue Book, p. 118)). We can work around this with some C code, but a more S-compatible solution might be a better idea.

Session-scope data is needed, basically, because the search path is session scope: When you quit, save the workspace and restart, the search path is not restored. This creates a problem for method dispatch. If the method search is not to go through each element of the search path on every call, looking for methods for this function, it has to build up a table of methods reflecting the currently attached libraries. If the search path is not restored, this table needs to be kept in some place that has scope of the current session, such as frame 0.


John Chambers<jmc@research.bell-labs.com>
Last modified: Fri Oct 13 15:29:51 EDT 2000