Background Proposed Research Experiments (Sample Sounds!)

EAR TO THE GROUND:
Hearing movement and change in the world
With or without our knowledge, our personal activities now leave behind complex traces: our Internet service providers record each stop we make across the Web; surveillance cameras monitor our passage through public places; credit card companies compile histories of our purchases and transactions. I want to create opportunities for sensory encounters with these electronic records, provoking questions as to the shifting boundaries between our public and private lives.

As an artist working in the medium of sound, I have long sought new ways of hearing inaudible phenomena and of mapping the observable world into the sound domain. I believe that the patterns and rhythms of human activity are worth listening to, and that the act of listening will reveal us to ourselves in new ways.

Ben Rubin

Statistics is the study of extracting usable information from data. Many of our analysis tools are visual, depending on graphical displays for "the revelation of the complex," as Tufte put it. But as we begin to examine large, dynamic systems like telecommunications networks, it is difficult to manually identify meaningful structures and patterns through simple visual interrogation. Our initial experiments gave web traffic a voice, and in so doing, uncovered interesting regularities in how people make use of the Internet.

Tukey wrote that the clever data analyst need only "expose himself to what his data is willing (or even anxious) to tell him." In some sense, Ben and I are attempting to make that happen, literally.

Mark Hansen

If the movements of each person browsing through Yahoo! right now made a sound, what would that sound like?

To most of us, the flow of information across the Internet is entirely abstract and intangible. When we browse a Web site, we are unaware of the thousands of people who are traveling alongside us, whose e-mail and Web pages flow in the same bit stream as our own. While it is beyond our capabilities to grasp the millions of simultaneous transactions taking place on the Internet, it is of compelling human interest to make sense of such environments in the large, to grasp the rhythms of our combined activities, of our comings and goings. Our inability to orient ourselves or otherwise fully perceive a larger environment is not a phenomenon unique to the virtual world. As communication and transportation technologies accelerate our movements and interactions, the spaces we live in are receding from our ability to directly sense them.

Sound provides a means to represent complex environments that makes their qualities accessible to our experience. While visual representations may be better for examining certain quantitative relationships, sound is unique in the way it communicates changes, rhythms and other dynamic characteristics.

We propose research and experiments to create new technologies for the transformation of information into sound. Our goal is to create "listening posts," points in the physical world or on the Web where people can tap in and listen to the pulse of global information systems.

Background

Music, as well as the visual arts, have a long history of creating shapes and compositions from observations of the natural world and of mathematics. Cage, Xenakis, and many others have based compositions on star charts, random data sequences, or structures from pure mathematics. In the scientific arena, efforts at representing information through sound date back to experiments in the 70's by John Chambers, Max Mathews and others at Bell Labs. These efforts involved either artificial data, derived from mathematical formulae, or static data sets. What makes this area of such interest now is the ability to compile in real-time, data sets that represent a broad spectrum of observable human activity and natural phenomena.

Within the last few years, passive data collection systems have begun to monitor not only the movements of people through Web sites and across the Internet, but also automobile and air traffic, transit systems, financial transactions, and more. Data from the natural world are also being tracked: weather patterns, seismic and oceanic activity, astronomical and cosmological data, and ecological indicators are all continuously logged. Advances in computation, communications and signal processing make it possible to link these data sources to sophisticated sound-generating systems and allow for interactive real-time, sound manipulation. Together, these advances open up the possibility of representing aspects of the world through sound that would have been impossible even five years ago.

Proposed Research

While the potential scope of this project is larger, our initial work will focus on data from computer networks. Within the four-month period of the AIM residency, we propose the following:

  • Identify interesting data that lends itself to auditory exploration;

  • Develop statistical and other techniques to isolate and amplify the expressive feature of the data;

  • Understand the musical and acoustical primitives that are most effective in experiencing information in this form;

  • Create flexible composition tools that allow us to interactively map dimensions of the data to the important acoustical primitives;

  • Develop a series of examples in which information can be heard.

Our work within the residency will culminate with the creation of several virtual listening posts to monitor characteristics of network traffic.

Beyond the residency period, larger goals include the creation of site-specific installations that focus on data that are in some way close to the site: a special environment in an airport terminal for listening to regional air traffic, a listening post high in a mid-town building where movements of cars on the streets below is transformed into a sonic composition, or a site on the San Andreas fault where seismic activity around the world can be heard – literally, an ear to the ground.

The Lucent resources that are critical to this project include:

  • Access to rich and varied sources of real-time data;

  • Ongoing critiques of our methods and results by statisticians, engineers and systems specialists, including John Chambers, a member of the original Bell Labs research team studying "Auditory Data Inspection" in the early 1970s.

  • Computational resources and expertise in extracting usable features from complex data.

The programming and audio tools we plan to use include the following: MAX, MSP, K2500, Csound, Perl S, C, and Matlab.

Preliminary Experiments

When we were first considering a collaboration, we met three times at EAR Studio and at Bell Labs to develop a proof-of-concept experiment. We studied a 24-hour sample of data taken from the November 18 access logs for www.lucent.com. As an initial experiment, we designed an auditory representation of the volume and depth of traffic as a function of five major topic areas within the site. The Lucent Web site is built hierarchically, in the sense that pages deeper in the site represent more detailed information than those at shallower levels. Visitors accessing the most general information affected the loudness and tonal balance of a low-register drone. Requests for content deep in the site are represented as higher-pitched pulsing tones: the faster the pulse, the more people are accessing that area, and the higher the pitch, the more detailed the content. At each level, each of the five divisions under study was assigned a unique tone, so that shifts in activity both within and between the areas could be heard. Listening to our two examples, one from the relatively quiet pre-dawn hours, and the other from the busy mid-afternoon, the sound reveals differences in the character of site usage at these times.

Samples
6:00 am The volume of traffic on the site is low. You can hear the actions of a handfull of people.
12:00 pm A period of increasing activity. Traffic will crest at a local maximum at 12:30pm.
2:30 pm The middle of the afternoon. The sounds record the most active period of the day for this site.