John M. Chambers
Bell Labs
Murray Hill, New Jersey
The project described today represents nothing less than a tour de force: successful assumption of responsibility for the quality and availability of a crucial information flow in the world's largest long-distance telephone network. To most of us, the job would appear daunting in prospect, not to mention terrifying. The sheer scale, as has been mentioned, would alone be challenging, along with the critical importance of the information. To compound difficulties, the project began from a large base of existing software systems dealing with pieces of the information flow. (At first glance, one might think the existence of prior software an advantage. One would not think so, however, if one had tried to get people to change their existing work habits in such circumstances.)
Despite such prior justification for pessimism, the project, as we have seen, was a success, and deserves the highest praise. As a starting point for discussion, let me suggest the following question:
How can we nurture similar successful interventions elsewhere?That we need to nurture similar projects seems undeniable. The complexity and importance of information flows in every range of activity, from commerce to government to science, is an inevitable byproduct of the technology underlying modern activities, from cheap high-speed computing to the Web. The current frenzy over data mining, while it may be subsiding slightly, is another recognition that understanding information flows is central to these activities. And ensuring the quality and accessibility of the data is essential to understanding.
Why was the present project a success? The first and most obvious reason (especially to your discussant) is the qualities that Rick Becker and Allan Wilks brought to the work: the ingenuity, thoroughness, and persistence with which they pursued their goals. Having worked extensively with the authors on the S language project, these qualities are very well known to me indeed, and I'm convinced they were key to the current success.
It would be tempting to say, though, that such special qualities are a requirement; that only exceptional circumstances can lead to successful intervention of this sort. Such a conclusion strikes me as both unacceptable and, overall, inaccurate. Unacceptable because the potential benefits are so attractive, and inaccurate because there are some steps we can take to make future projects less forbidding.
The most obvious benefit comes simply from improved data quality. Quantifying that benefit will depend on circumstances; my impression is that AT&T estimated the benefits as very substantial indeed in the present case. It's worth reflecting, at the same time, on some subsidiary benefits; in particular, that of greater accessibility. To master data quality (anywhere, but perhaps particularly in transaction situations), one needs a clear and well-defined formulation of the process--its stages and the concepts that define what happens at each stage. In other words, a model of the data flow and data structure that makes sense, that corresponds as well as possible to what really happens, and that can be implemented in terms of access and control.
Having such a model, and in particular having a corresponding implementation in software, allows us to use it beyond the immediate task of data quality. Not only is data quality key to allowing us to get the most from the information, the same requirements needed to achieve the quality will often open up the data to us in a more effective way.
So if we believe that successful projects to achieve data quality and data accessibility are important, are there any general steps we can take to nurture such projects? Let me suggest two, as points for discussion:
The first point may be regarded by some as mainly an issue of attitude and administration, as opposed to technology. The assertion is that one needs to understand the how and the why of the processes that generate and transform the data. Otherwise, intervention by the likes of us is unlikely to be very effective. Part of this is essentially the art of collaborative applied statistics--knowing the who of the project is important as well. Collaboration, to emphasize the point, and not consultation--a consultant too easily becomes a hired expert doling out advice by the hour. Proper collaboration requires more investment in time and study. That investment is often rewarded, however, first in more effective contributions to the application. In addition, a continuing collaboration can and should result in greater reuse of the insights gained, to borrow from the term software reuse, currently in vogue in computing. In fact, the consulting reuse will be helped if the software itself is accessible for reuse (something as simple as keeping the automatically generated logs of a statistical system would be a start).
As mentioned, such principles are relatively well known and moderately widely taught as part of the practice of statistics. Outstanding practitioners have eloquently endorsed them (John Tukey, George Box, and David Cox come to mind in particular). Numerous success stories from various organizations represented in the present symposium support the value of such understanding, I believe. (The non-success stories provide insights too, but understandably we may choose not to discuss those, at least for quotation.) All the same, I think the principles deserve all the repetition they can get. For issues of data quality, in particular, the collaborative role of statistics is crucial to taking the longer, process-oriented view that is required.
As a slightly more technological comment, and perhaps to stimulate some discussion, let me suggest that statistical models still need to move towards a focus on the process. It's only rarely that our formal thinking extends beyond specific sets of data to an underlying, multi-stage process of which the data are results. For example, in many transaction applications there are choices about which potential sources of data one chooses to pursue--at which stages will we implement ``taps'' and what access requests will we formulate? The tradeoffs are important, but if our conceptualization doesn't go beyond the specific datasets, we will have difficulty grappling with the alternatives either before or after we make specific choices.
It's the second of the two points, however, that relates to some particularly relevant current research directions (and that provides the opportunity, dear to any discussant's heart, to bring in some of my own current preoccupations). Let me focus on one aspect of the right tools: software that gives us easy and powerful access to the information flow. Without that, practical influence on data quality and appropriate access to the information will be hard to achieve.
In order to implement a substantial project in improving data quality and accessibility, we must go to the process. We must get to the sources of the data, not wait for the data to arrive in our own cosy statistical environment. We must be ``out there'' and not just ``in here''. In particular, we cannot deal exclusively with our favorite statistical system or whatever software we have chosen as our home base. All sorts of software systems will be involved--database management, communication, data acquisition, user interfaces, plus an unpredictable range of software related to the particular project.
We need hooks into this software. More, we need ways to communicate between particular software systems. To understand issues of data quality, we need to visualize and model at all stages of the process. This requires either bringing information about the data out to suitable software, or bringing the software into the environment managing the data.
By no coincidence, software supporting both possibilities is part of some exciting current research in statistical computing. One key aspect is the design of new inter-system interfaces. These interfaces provide a model in which objects in one system are made available to another system at a level of abstraction allowing far more powerful sharing of software than earlier interfaces, which are largely text-based and lacking object structure.
The distinction being made here may seem subtle, but it has an analogy to the distinction between consulting and collaboration made earlier! We're trying to allow the software systems to collaborate, in the sense of sharing their internal structures, rather than just communicating in a trivial, external fashion.
Let's take just one example. Suppose one of the systems is a statistical language (you may guess that I have one in mind). At any rate, within this language we can express and fit models to data, e.g., linear regression. The fitted models are objects within the language; in particular, they have some structure that lets the user manipulate them. Information can be extracted (coefficients, fitted values, residuals, and perhaps more complex information as well). The model can be modified and refitted.
Now suppose we want to make that model available via an interface, say to a database management system (DBMS). If all we can do is communicate by strings, we (as users) will have to figure out how to express the statistical commands in the DBMS, tell the statistical language to print some results in a format the DBMS can read, and then read the results back in. But the real structure of a statistical model cannot be effectively reduced to such triviality. The better way, and the focus of current research, is that the statistical system makes the model available, inside itself. It advertises a set of methods or requests that it will honor. In effect, these constitute a definition of the model, at a level of data abstraction that makes the model truly available to other systems.
Achieving collaboration at this level requires sharing a suitably advanced way of describing objects. There are a number of implementations, and no one of them is uniquely suitable. One very general approach uses the IDL language defined as part of the CORBA standard for system communication is one mechanism. CORBA interfaces to many of the relevant systems exist, including interfaces to implementations of the S language.
To go further into details would get a long way beyond a discussion leader's prerogative, so let me just offer a pointer to some of the relevant work, part of the Omega project, a joint research effort in statistical computing (see, for example, the project's web site).
This document was generated using the LaTeX2HTML translator Version 99.1 release (March 30, 1999)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -split 1 paper
The translation was initiated by John Chambers on 2000-11-29