Ear to the Ground: sonifying communication systems
Abstract
In Ear to the Ground, a statistician (Hansen) and a media artist (Rubin) investigate the application of statistical methods and sound-design principals to the sonification of real-time information from communication systems. This paper presents some preliminary results from the sonification of browsing activity on Lucent's web site, as well as the sonification of a large number of chat sites in real-time.
1. Introduction
2. New Data, New Challenges
Modern work in sonification emerged in the literature on computer-human interfaces and has over the last decade matured into its own field of scientific inquiry. The use of sound in exploring the information hidden in data, the principles and broad application of auditory displays are eloquently described in Kramer (1994). Early application areas included real-time monitoring of financial data, medical diagnostics, and even air traffic control systems. Computer simulations also provided extensive data for sonification. Since that point, a virtual explosion has taken place in our ability to collect data relating to human communication and social systems.
Today, almost every aspect of our lives can be "rendered" digitally. Advances in data collection technologies have made commonplace continuous, high-resolution measurements of our physical environment (weather patterns, seismic events, ecological indicators). Equally open to observation are our routine movements through and interactions with our physical surroundings (automobile and air traffic, large-scale land use). In computer-mediated settings, our activities either depend crucially on or consist entirely of complex digital data (financial transactions, accesses to global information systems, Web site and Internet usage). As a reflection of the diversity and variety of the systems under study, these data-based descriptions of our daily lives tend to be massive in size, dynamic in character, and repleat with rich structures.
The advent of these enormous repositories of digital information presents us with an interesting challenge: How can we represent and interpret such complex, abstract and socially important data? In a new collaboration, Ear to the Ground, we have begun an exploration into ways of creating experiential encounters with otherwise abstract data streams, especially through sound [1].
3. Examples
3.1 Web site traffic
Every day, large web sites like Yahoo attract hundreds of thousands of visitors. During active periods, there can be thousands of people accessing data simultaneously from the web site. For many information portals like Yahoo, the speed of the server provides us with our only clue about the number of other people accessing information. Could sound be used to create an ambient display that characterized the activity on the site?
To find out, we took 24 hours of page request data from Lucent's web site,
http://www.lucent.com. The data contains a record of every page request ("hit") on Lucent's site during the 24 hour period. Each data record contains (among other information) the page requested, the IP address of the requestor, and a time stamp.The Lucent Web site is built hierarchically, in the sense that pages deeper in the site represent more detailed information than those at shallower levels. Visitors accessing the most general information affected the loudness and tonal balance of a low-register drone. Requests for content deep in the site are represented as higher-pitched pulsing tones: the faster the pulse, the more people are accessing that area, and the higher the pitch, the more detailed the content. At each level, each of the five divisions under study was assigned a unique tone, so that shifts in activity both within and between the areas could be heard. Listening to our two examples, one from the relatively quiet pre-dawn hours, and the other from the busy mid-afternoon, the sound reveals differences in the character of site usage at these times.
We wanted our display to convey qualitative information about the site usage, such as:
To do this, we created a mapping as follows:
|
Overall activity |
Low-register drone; five pitches in the drone represent the five content areas of Lucent's site. The aggregate number of recent hits within each of the five areas modulates the loudness of each of the five pitches. |
|
Mid-level browsing |
Overall activity within the site is mapped to low-register drones.
3.2 Chat rooms and bulletin boards
Increasingly, the web is becoming a source of dynamic, real-time data streams. There are considerable difficulties in navigating this content. This should not be surprising given that search functions for finding static content on the web can be a usability nightmare. With these dynamic data streams, we have a direct and compelling view of people's activities. Unlike the web traffic example, the difficulty in sonifying these sources forces depends on the number of streams we choose to explore. Data reduction through statistical modeling becomes critical. In addition, because users will likely be interacting with an audio display for analysis purposes (finding a good topic) the aesthetic component becomes even more important as well.
4. Collaborative research and the design of audio displays degree to which aesthetic are critical to functional success.
The two applications outlined in this paper are the first outcomes of a collaboration sponsored by Bell Laboratories and the Brooklyn Academy of Music under the Arts in Multimedia project (AIM). The goals of AIM is to bring together researchers (in this case a statistician) and artists (in this case a sound artist). We believe that the sonification work outlined in this paper differs not only in the data sources considered, but also in our approach.
5. Conclusion and future work