"The audiences would be the artists
and their life would be the arts."
By Mark Hansen
and Ben Rubin
To most of us, the
transmission of information across the Internet is entirely abstract
and intangible. But in 1995, artist Natalie Jeremijenko quite
literally tapped into the bit stream coursing through the local
network at Xerox PARC, transforming the flow of data into a physically
perceptible phenomenon. Her installation, titled "Dangling String,"
consists of a long plastic fiber that is attached to an electric motor
and mounted on the ceiling. Each bit traveling across a stretch of
nearby Ethernet cable triggers the motor. During periods of heavy
network activity, the string whirls rapidly; but when traffic is
light, the motor twitches sporadically, almost
undetectably. Jeremijenko's elegant and accessible piece represents an
important step in the quest by artists to reveal the mostly invisible
realm of electronic data, a world that is increasingly astonishing in
its size and nature.
Today, almost every aspect of our lives can be "rendered"
digitally. Advances in data collection technologies have made
commonplace continuous, high-resolution measurements of our physical
environment (weather patterns, seismic events, ecological
indicators). Equally open to observation are our routine movements
through and interactions with our physical surroundings (automobile
and air traffic, large-scale land use). In computer-mediated settings,
our activities either depend crucially on or consist entirely of
complex digital data (financial transactions, accesses to global
information systems, Web site and Internet usage). As a reflection of
the diversity and variety of the systems under study, these data-based
descriptions of our daily lives tend to be massive in size, dynamic in
character, and repleat with rich structures.
The advent of these enormous repositories of digital
information presents us with an interesting challenge: How can we
represent and interpret such complex, abstract and socially important
data? In a new collaboration, Ear to the Ground, we have begun
an exploration into ways of creating experiential encounters with
otherwise abstract data streams, especially through sound [1]. As part of
our initial efforts, we have gathered several examples of what we will
term "data art," works produced by artists, designers and engineers
that challenge our traditional notions of information display. These
pieces each have an aesthetic function, but because of their reliance
on data, they also serve as a basis for forming inferences about the
phenomena they describe.
The phrase we borrowed as the title for this article, "The
audiences would be the artists and their life would be the arts," is
from a comment made in 1966 by pianist Glenn Gould, who was
speculating on the nature of art in the electronic age. Gould imagined
the possibilities of involving the audience, and information about
them, in the process of creating and performing works of art. What
makes this statement compelling now is the fact that so much
information is readily available about the public, Gould's
audience/artists.
Music and Mathematics
Musicians, as well as visual artists, have a long
history of creating shapes and compositions from observations of the
natural world. In some cases, these works offer us a new way to
experience physical phenomena. While many composers have reveled in
nature's regularity, say in the cycles of planetary orbits, others
have attempted to capture the scale and force of nature. Avant-garde
composer John Cage, for example, randomly generated compositional
elements to imitate the "character" of nature. In the 1950s, composer
Iannis Xenakis applied the term "Musique Stochastique" to music
characterized by "masses of sounds"; systems consisting of so many separate
elements that we can only hear the effect of the "whole," but the contributions of
any single individual cannot be determined precisely
[2]. Xenakis motivates this idea by describing the sonic phenomena of a
"crowd of dozens or hundreds of thousands of people" -- each one an
individual, listening, reacting, and producing sound, propagating
chants and cheers in graceful, chaotic waves [3].
While Xenakis's image was inspired by the mid-century
political demonstrations around him, groups of this size gather daily
on large Web sites like Yahoo! or CNN.com. To what extent is it
possible to represent these voiceless, virtual crowds through sound?
How do we make perceptible their activities, their movements? And in
so doing, what new insights do we form about the nature of Web
traffic? This is the starting point of our project, Ear to the
ground. While we have focused initially on Internet usage, our
broad goal is to "sonify," or make audible, digital transcripts of
human activity. As a collaboration between a statistician (Hansen) and
an artist (Rubin), we are looking to create sound representations that
further the arts of data analysis, discovery, and
expression.
In the 1980s and 90s, Xenakis became interested in a more
mathematically-derived form of composition to produce what he thought
were forms of classical perfection. In 1991, he created a computer
program that generates a complete musical piece by pure algorithmic
computation. Xenakis is not alone in his use of music or sound as a
means of experiencing concepts in pure mathematics. Daniel Cummerow,
for example, has several compositions based on the digits of pi and
other well known sequences [4].
Lars Kinderman's MusiNum (the Music of the
Numbers) is an example of "fractal music"; a number-theoretic
algorithm generates an entire musical score, a "self-similar" series,
from a single integer [5]. Applying this idea, Kinderman's Web site will
construct a four-part musical composition based on the separate digits
of the IP address of a visitor's computer.
Data Art
In considering our Ear to the Ground project, we
have come to understand an important distinction between sonic
compositions that are mathematically derived and those that are
data-driven. The former amount to transformations of mathematical
structures, created or selected by the composer, into the sensory
domain of sound. Data-driven compositions, on the other hand, take
streams of measurements as their basis. Both math-derived and
data-derived music have in common the need for sonification methods,
the direct mechanism by which the numbers, whether algorithmically generated
or empirically measured, are rendered audible. A primary goal in our
Ear to the Ground effort is to establish methods that preserve and
reveal the inherent structure and dynamics of the data. The selection
of the data -- deciding what to measure and how to measure it --
becomes a central issue in creating data art. With Ear to the
Ground, we are looking initially to data on human behavior as
expressed in patterns of Internet usage -- data that will potentially
have the same kind of rich, dynamic patterns that Xenakis found so
compelling in his crowd imagery.
The Life Music compositions of John Dunn (an artist) and
Mary Anne Clark (a biologist) are another example of data art [6]. The
subject of their "sonification" experiments is the basic component of
all life, DNA. The Human Genome Project (HGP) has now "sequenced"
roughly 65% of the 3 billion chemical base pairs that make up human
DNA, and at the current rate, a complete picture of our 100,000 genes
will be available by 2003. The HGP poses a number of ethical, social
and legal questions that are already filtering into public forums. By
giving a voice to strands of protein in human DNA, the compositions by
Dunn and Clark are a concrete point of contact with this rich data
set. Their goal is to convey information about the primary amino acid
sequence (mapped to pitch), as well as the folding pattern (mapped to
separate instruments). Life Music stands as an example of music being
directly inspired by the audience, a literal and detailed fulfillment
of Gould's statement.
In as much as it retains "fidelity" to data streams, the
concept of data art has useful precedents in the scientific community
as well. Recently, Andrew Michael of the US Geological Survey "played"
the seismograms recorded from earthquakes in the Los Angeles area [7]. His
audio representations allow the listener to discriminate between
earthquakes based on their distance from the siesmogram. A number of
other features about earthquakes can be derived from their
sound. Michael's work can be seen as an extension of earlier ideas
presented by Speeth in 1961, who listened to seismograms in the hope
of detecting differences between earthquakes and underground bomb
blasts [8]. These examples barely scratch the surface of scientific
applications of sound to help investigate data. An excellent survey of
this area is available through the ICAD (International Community for
Auditory Display) Web site [9].
"A revelation of sound even where we don't expect that
it exists"
The quote above by John Cage (taken from [10])
reflects his insatiable desire to hear everything -- even
things that are not intrinsically audible. In 1938, John Cage became
fascinated by the new technology of optical sound reproduction
(transforming sound into light modulations in order to record it
photographically onto motion picture film) that had
revolutionized the motion picture industry. Cage pushed the idea of
sound-as-light further, observing in his Future of Music: Credo
that "Any design repeated often enough on a soundtrack is
audible. Two hundred and eighty circles per second on a soundtrack
will produce one sound, whereas a portrait of Beethoven repeated fifty
times per second on a soundtrack will have not only a different pitch
but a different sound quality" [11]. Cage understood that the new
technology had created a way not only of recording sound and playing
it back, but also of transforming other kinds of information
images, for example into sound.
Carrying the idea in a different direction, Cage went on
to imagine ways to amplify and make audible the molecular and atomic
vibrations of a static, inanimate object, such as an
ashtray. Experienced in this way, how would the sound of an ashtray
differ from the sound of a piece of wood, or a stone, or a grapefruit?
Could sound somehow help us fathom the true nature of each of these
things? Consider again the seismic data of Speeth and Michael. While
the events themselves (earthquakes) have an intrinsic sound, the data
recorded from these events do not. Creating sound from these data
offer us a new experience. If, in our sonic rendering, we preserve the
character of the data, then we can draw conclusions about different
types of events.
By why stop at sound? We rely on many different sensory
inputs for collecting information from the natural world besides sight
and sound. Hiroshi Ishii, who founded the Tangible Media Group at the
MIT Media Lab, is exploring how physical phenomena can be used to
construct new modalities or representations for experiencing data [12]. In
a project called Tangible Bits, Ishii and his collaborators
follow Jeremijenko's lead, using physical kinetic motion, streams of
air, and other "media" to construct ambient data displays [13]. In one
project, the group linked multiple streams of data to large, rotating
pinwheels that spin faster or slower depending on fluctuations in the
data sources. Perceptible in one's peripheral vision, or merely by the
sensation of the gentle breezes it stirs, the Pinwheel
apparatus is an effective means of communicating a non-specific sense
of activity and change.
Listening Post
As we become more adept at observing, measuring, and recording both
our physical and virtual movements, we can expect to see a
proliferation of large, complex data sets. These new digital records
of human activity often force us to consider difficult societal
questions. From the current debate over privacy on the Internet, to
ethical concerns over the use of genetic information, we recognize
that the now simple act of compiling data has serious political
implications. To further complicate matters, the scale and structure
of these data make them difficult to comprehend in the large,
relegating activities like data analysis to a select few. Even then,
experiencing and appreciating what the data have to say is a difficult
task, stretching current methods and tools to their limits.
The notion of data art offers to the public a means of
engaging these often abstract measurements.
In our Ear to the Ground collaboration, for
example, we plan on establishing "listening posts," points in the
physical world or on the Web where people can tap in and listen to the
pulse of global information systems. In the same way, Dunn and Clark
provide us a unique impression the human genome, a view that in their
words offers "empirical proof of the esthetic patterning of nature's
deep structure" [14]. Taken farther, pieces of data art also serve the
seasoned analyst, highlighting features of the data that might have
been difficult to ascertain otherwise. Ishii's ambient displays, for
example, allow for passive monitoring of often remote data sources, a
novel approach to statistical process control.
Throughout our discussion, we have stressed
the need for data art to provide an impression that is in some sense
true to the character of the source. By interacting with these pieces,
we inevitably engage in some form of data analysis, albeit informal.
We form questions, draw conclusions, and make inferences based on an
experience of the data that might involve many senses at once. How do
we gauge the effectiveness of these new displays? When are they
trivial or misleading and when are they likely to spawn new insights?
This is certainly new territory for both statistics and the arts. Our
success in making the most of these new ideas will depend on
collaborative efforts like Ear to the ground, projects that
combine empirical science with artful media.
References