MSRI Workshop on Nonlinear Estimation and Classification
Berkeley, California
March 19-29, 2001

· Download a Workshop Poster, PDF format
· Visit the MSRI workshop page

Goal

Researchers in many disciplines face the formidable task of analyzing massive amounts of high-dimensional and highly-structured data. This is due in part to recent advances in data collection and computing technologies. As a result, fundamental statistical research is being undertaken in a variety of different fields. Driven by the complexity of these new problems, and fueled by the explosion of available computer power, highly adaptive, non-linear procedures are now essential components of modern "data analysis," a term that we liberally interpret to include speech and pattern recognition, classification, data compression and signal processing. The development of new, flexible methods combines advances from many sources, including approximation theory, numerical analysis, machine learning, signal processing and statistics. The proposed workshop intends to bring together eminent experts from these fields in order to exchange ideas and forge directions for the future. It also intends to introduce the research topics to graduate students by providing travel support and by requiring the last speaker of each session to give an overview of the field.

A Brief Survey of the Area

Three ingredients are common to the new class of non-linear methods:

  1. Approximation spaces (wavelets, splines, neural networks, trees) or other simple probabilistic structures as building blocks for representations of statistical objects;
  2. Algorithms for identifying the "best" representation or combining several "promising" candidates; and
  3. Statisticalframeworks to judge between competing representations.

A key component in recent non-linear modeling efforts is the design of efficient representations for (complex) objects such as curves, surfaces and images. Here, efficiency often relates to the number of descriptors (i.e. basis functions or general predictors). This definition has parallels with recent results in approximation theory on n-term expansions and best basis algorithms. Similarly, in machine learning, the focus is on repeatedly applying weak-learners or relatively simple representations like the sigmoid function used in (artificial) neural networks or very shallow trees, known as stumps. On the other hand, new tools for image and texture analysis draw their representations from the human visual system, where efficiency might be evaluated in biological terms. Finally, graphical or hidden Markov models represent a distribution involving complex dependencies by connecting simple (conjugate) distributions in a hierarchical manner.

In statistical problems, the search for the single "best" representation among a (potentially vast) number of competitors is really a problem in model selection. Applying these ideas in the context of approximation spaces, for example, has led to new wavelet shrinkage schemes and new methods for free-knot splines. Various selection criteria have been suggested in these problems, each with differing strengths and weaknesses, but none dominating all the others over a wide range of problems. Even with an agreed selection criterion, difficulties can occur in identifying the "best" representation as it is often a single element in an extremely dense candidate set which cannot be exhaustively searched. Efficient optimization methods need to be devised to overcome this problem.

Rather than focusing on a single model, recent empirical and theoretical results have shown that we can achieve improved predictive performance by combining several competing representations. The Bayesian paradigm naturally leads to a framework for averaging models, and has been applied extensively in the context of approximation spaces. From a theoretical standpoint, these techniques depend quite strongly on the underlying prior assumptions, and understanding their performance is an area of active research. Other techniques for combining models have arisen in the machine learning literature, most notably boosting and bagging, and have proved to be extremely effective in practice.

When working with massive amounts of data, a test or hold-out data set has been commonly used for judging competing procedures. The question of which evaluation criterion or loss function to use still remains. Some loss functions are more sensible than others in a particular problem, although the squared loss has been the most conventional. With the current computing power, it is believed possible to use more sensible loss functions, for example, the absolute loss for image processing. This, however, needs a consented effort of the research community to deviate from the norm. The proposed workshop is an ideal place to reach such a consensus.

· Winter, 2002
Springer-Verlag has published the proceedings! View the Table of Contents. Buy it from Amazon.com!
· Fall, 2001
MSRI produces streaming videos of the talks.
View the Talks.
A preliminary listing of the invited portion of the program.

Updated: 8/27/00
A preliminary listing of the contributed portion of the program.

Updated: 01/26/01
· David Denison, Imperial College
· Mark Hansen , Bell Labs
· Chris Holmes, Imperial College
· Robert Kohn , Univ. of New South Wales
· Bani Mallick, Texas A&M
· Martin Tanner , Northwestern
· Bin Yu, UC Berkeley
· MSRI workshop page
The MSRI has made information about funding, lodging and other local arrangements available at this site.
· Format of the Program
· Proceeding via Springer-Verlag
· Funding Sources