Introduction

Computer users are becoming increasingly familiar with the idea that two or more applications can be in existence and performing tasks concurrently. Individuals can be writing a document in a word processor, reading mail and extracting data from another application without having to close one application and start another as they move between the applications. As statisticians, a common paradigm is to be performing lengthy computations in the background while doing other more immediate work interactively. An environment that supports this notion of concurrent applications is called multi-tasking and supports two or more processes (instances of an application) running at the same time and competing for resources.

Users interact with an application in order to accomplish some overall task. This task might be divided into several sub-tasks, some of which must be completed in a specific order and others that can be done at any stage in the overall task. For example, an initial statistical analysis of a dataset might involve

  1. reading the data into the application
  2. computing a variety of summary statistics and plots
  3. exploring some modelling approaches for the data
  4. examining the output and diagnostics of fitted models
  5. pursuing other statistical estimators for a subset of the fitted models.
The task is this whole analysis procedure made up of these steps. The steps are typically performed in the order specified and perhaps a sub-sequence iterated. However, within each task, as described, the order may not be important. (Perhaps this should go first and then introduce multi-tasking and then onto multi-threading). While the operating system may support multiple processes, few of the these applications have the capability of performing multiple within-application tasks simultaneously. Applications viewed through a graphical user interface (GUI) often appear as if they are performing two tasks simultaneously, but frequently this Thread Definition. C Implementation.

Uses for Multi-Threading

Multi-threading is desirable in at least two general contexts.
  1. Efficiency through parallelization of a task
  2. Event handling in GUIs and client-server scenarios.
A third more specific scenario where multi-threading leads to increased run-time efficiency arises when one task is blocked from input/output operations through competition for the resource (e.g. disk, etc.). In this case, other tasks can proceed when a single-threaded application would block until the resource was available.

Examples

Cross-Validation
Bootstrapping
Simple Image Registration
A common problem in the different forms of statistical image analysis involves working with a sequence of images.
Data Acquisition from a Device
Remote Databases
Markov Random Field Simulation and Monitoring

Parallel, Distributed and Multi-Threaded Applications

Computer Science and other disciplines have differentiated between what are termed distributed and parallel computation and similarly between concurrent and parallel. Parallel computing typically refers to performing multiple tasks on the same machine. The standard meaning for th There are several flavors of application concurrency.

Difficulties with the Multi-Threaded Approach

In spite of the potentially large benefits, statisticians and other disciplines have been relatively reluctant and slow to harness the multi-threaded paradigm.

Persistence

Programming

Synchronization
Case-by-case solutions. Program validation.
Portability

Availability

Previous inaccessibility and specialization. Within the past two or three years, multi-threaded applications and libraries have become more common and accessible to the casual user.
Java
POSIX
PVM
Hardware

Duncan Temple Lang<duncan@stat.Berkeley.EDU>
Last modified: Sat May 31 07:31:23 PDT 1997