Goal
Researchers in many disciplines face the formidable task of analyzing
massive amounts of high-dimensional and highly-structured data. This
is due in part to recent advances in data collection and computing
technologies. As a result, fundamental statistical research is being
undertaken in a variety of different fields. Driven by the complexity
of these new problems, and fueled by the explosion of available
computer power, highly adaptive, non-linear procedures are now
essential components of modern "data analysis," a term that we
liberally interpret to include speech and pattern recognition,
classification, data compression and signal processing. The
development of new, flexible methods combines advances from many
sources, including approximation theory, numerical analysis, machine
learning, signal processing and statistics. The proposed workshop
intends to bring together eminent experts from these fields in order
to exchange ideas and forge directions for the future. It also
intends to introduce the research topics to graduate students by
providing travel support and by requiring the last speaker of each
session to give an overview of the field.
A Brief Survey of the Area
Three ingredients are common to the new class of non-linear methods:
- Approximation spaces (wavelets, splines, neural networks, trees)
or other simple probabilistic structures as building blocks for
representations of statistical objects;
- Algorithms for identifying the "best" representation or
combining several "promising" candidates; and
- Statisticalframeworks to judge between competing
representations.
A key component in recent non-linear modeling efforts is the design of
efficient representations for (complex) objects such as curves,
surfaces and images. Here, efficiency often relates to the number of
descriptors (i.e. basis functions or general predictors). This
definition has parallels with recent results in approximation theory
on n-term expansions and best basis algorithms. Similarly, in
machine learning, the focus is on repeatedly applying weak-learners or
relatively simple representations like the sigmoid function used in
(artificial) neural networks or very shallow trees, known as stumps.
On the other hand, new tools for image and texture analysis draw their
representations from the human visual system, where efficiency might
be evaluated in biological terms. Finally, graphical or hidden Markov
models represent a distribution involving complex dependencies by
connecting simple (conjugate) distributions in a hierarchical manner.
In statistical problems, the search for the single "best"
representation among a (potentially vast) number of competitors is
really a problem in model selection. Applying these ideas in the
context of approximation spaces, for example, has led to new wavelet
shrinkage schemes and new methods for free-knot splines. Various
selection criteria have been suggested in these problems, each with
differing strengths and weaknesses, but none dominating all the others
over a wide range of problems. Even with an agreed selection
criterion, difficulties can occur in identifying the "best"
representation as it is often a single element in an extremely dense
candidate set which cannot be exhaustively searched. Efficient
optimization methods need to be devised to overcome this problem.
Rather than focusing on a single model, recent empirical and
theoretical results have shown that we can achieve improved predictive
performance by combining several competing representations. The
Bayesian paradigm naturally leads to a framework for averaging models,
and has been applied extensively in the context of approximation
spaces. From a theoretical standpoint, these techniques depend quite
strongly on the underlying prior assumptions, and understanding their
performance is an area of active research. Other techniques for
combining models have arisen in the machine learning literature, most
notably boosting and bagging, and have proved to be extremely
effective in practice.
When working with massive amounts of data, a test or hold-out data set
has been commonly used for judging competing procedures. The question
of which evaluation criterion or loss function to use still remains.
Some loss functions are more sensible than others in a particular
problem, although the squared loss has been the most conventional.
With the current computing power, it is believed possible to use more
sensible loss functions, for example, the absolute loss for image
processing. This, however, needs a consented effort of the research
community to deviate from the norm. The proposed workshop is an ideal
place to reach such a consensus.
Organizing Committee
- David
Denison
(Imperial College)
- Mark Hansen
(Bell Labs)
- Chris Holmes
(Imperial College)
- Robert Kohn
(Univ. of New South Wales)
- Bani
Mallick (Texas A&M)
- Martin
Tanner (Northwestern)
- Bin Yu
(Bell Labs and UC Berkeley)
Funding
In addition to funding from the Mathematical Science s Research Institute,
proposals are being prepared for the NIH (US), NSF
(US), and the NSA (US). In consultation with MSRI, we will also
seek support from corporate sources.
Proposed Time and Format
The workshop will be held at the
Mathematical Sciences Research
Institute (MSRI) in Berkeley, California during the Spring of 2001.
This workshop is by definition an interdisciplinary affair. As such,
we have attempted to give each area sufficient representation so that
new and important developments can be presented and discussed. In
addition, we hope to solicit a number of contributed talks and posters
in the separate disciplines, mainly from new researchers. Given the
tremendous activity planned, we have alloted two weeks for the
workshop.
- The workshop will consist of 2 sessions per day, one invited and
one contributed.
- Morning discussion sessions will run from 9am to 12:30pm, and
consist of four 45 minute talks with a 30 minute break. The final
speaker will be asked to include an overview of the field (including
the work of the other speakers).
- Special organized lunches will be planned to help facilitate
further discussions based on the morning's presentations.
- Afternoon sessions will run from 3 to 6:30pm, and will be made
up of eight 20 minute short talks with two 15 minute breaks. Short
talks will be given primarily by new researchers.
- Ongoing posters from new researchers and graduate students will
be on display each day near the main meeting room.
Speakers and Sessions
Below we list a number of rough, proposed sessions. The speakers
listed constitute the organizers' wish list.
Those speakers who have agreed to (at least tentatively)
participate appear in bold face in the sessions below.
These sessions will
undoubtedly change when we receive abstracts from each speaker. At
that point, we anticipate reshuffling the sessions to better focus the
separate ideas.