Brooklyn Academy of Music: Arts in Multimedia Project

EAR TO THE GROUND:
Hearing movement and change in the world

Experiments with Chat Rooms and Bulletin Boards

In this example, we present an auditory display for three chat rooms, each concerned with a different topic. For the sonification, each different room is assigned a different pitch group, reinforcing subject changes when they occur. For each room, we build three streams of information:

The topics are spoken by a TTS system at regular intervals in a pitched monotone, and are panned alternately hard left and hard right in the stereo field, creating a sort of rhythmic "call and response." The sample sentences are panned center, and rendered with limited inflection (as opposed to the pitched monotone of the topics). The tonal, rhythmic and spatial qualities of the topics contrasts sufficiently with the sample sentences to create two distinctly comprehensible streams of verbal information.

The entropy vector controls an algorithmic piano score. When entropy is minimal and the discussion in the chat room is very focused on one subject, chords are played rhythmically in time with the rhythmic recitation of the topics. As entropy increases and the conversation diverges, a Gaussian distribution is used to expand the number, range and dynamics of notes that fall between the chords. With this audio component, one can easily differentiate a well-moderated content source from a more free-form, public chat without distracting from the TTS annotations. The piano score also serves a secondary function as an accompaniment to the vocal foreground, enhancing the compositional balance and overall musicality of the sound design.

A more formal description of the mechanism used to sonify these data is given in our ICAD 2001 paper.

Samples [Created 8/00]
Chat Sample Three separate topics are contained in one file.
Timestamp:
00:00.00-00:01.47
The first portion of this sample concerns the recent recall of Bridgestone tires and was based on a www.cnn.com chat room. This discussion was heavily moderated and hence the backing piano score frequently reduces to a simple rhythm.
Timestamp:
00:01.47-00:03.21
For our second topic we recorded chat exchanges on www.financialchat.com one morning when Yahoo's stock opened low. In this example, we hear day traders frantically exchanging predictions about when Yahoo's stock will ``bounce.''
Timestamp:
00:03.21-end
The final topic in this sample is again from www.cnn.com and treats a recent strike by the Screen Actor's Guild and the American Federation of Television and Radio Artists. This chat room was much less moderated than the previous CNN chat, and the backing piano score reflects that.



Yahoo!/Bounce appears on compilation CD assembled by Elliot Sharp. You can buy a copy at CDemusic.