by Paul Vickers
Description
This chapter looks at a range of auditory display and sonification applications that have tackled the problem of monitoring real-time data streams and concludes with some recommendations that further research should be informed by semiotic and aesthetic thinking and should explore the use of soundscapes, steganographic embedding, model-based sonification, and spatialization as profitable techniques for sonifying monitoring data.
Download Chapter
Download the chapter: TheSonificationHandbook-chapter18 (PDF, 1.5M)
Media Examples
Example S18.1: stormyWinter
Example of information pull scenario. A nine-dimensional set of weather data was sorted by region and then allocated to a weather ‘prototype’. This example is a sonification of a stormy winter day with snow.
media file S18.1
download: SHB-S18.1 (mp3, 241k)
source: Thomas Hermann, Jan M. Drees, and Helge Ritter. Broadcasting Auditory Weather Reports A Pilot Project. In Eoin Brazil and Barbara Shinn-Cunningham, editors, ICAD ’03 9th International Conference on Auditory Display, pages 208211, Boston, MA, 2003. ICAD.
http://www.techfak.uni-bielefeld.de/ags/ami/datason/demo/ICAD2003/wav/winter_snow_storm.mp3
Example S18.2: humidSummer
As S18.1, but this time the weather vector is of a hot and humid summer day with a thunder storm imminent.
media file S18.2
download: SHB-S18.2 (mp3, 241k)
source: Thomas Hermann, Jan M. Drees, and Helge Ritter. Broadcasting Auditory Weather Reports A Pilot Project. In Eoin Brazil and Barbara Shinn-Cunningham, editors, ICAD ’03 9th International Conference on Auditory Display, pages 208211, Boston, MA, 2003. ICAD.
http://www.techfak.uni-bielefeld.de/ags/ami/datason/demo/ICAD2003/wav/summer_storm.mp3
Example S18.3: Sonnet
This pair of sonifications shows David Jameson’s programming language sonification Sonnet being used to sonify a faulty bubble sort algorithm. The second example in the system illustrates how the audio is used to infer the presence of an indexing bug.
media file S18.3
download: SHB-S18.3 (mp3, 2M)
source: David H. Jameson. Sonnet: Audio-enhanced Monitoring and Debugging. In Gregory Kramer, editor, Auditory Display, volume XVIII of Santa Fe Institute, Studies in the Sciences of Complexity Proceedings, pages 253265. Addison- Wesley, Reading, MA, 1994. The sound example was taken from the CD that accompanied the book.
Example S18.4: Listen
This example shows a bubble sort algorithm being sonified using the Listen system. Note the use of bell sounds to signal the start and end of the loop and the woodblock which denotes each iteration.
media file S18.4
download: SHB-S18.4 (mp3, 1.9M)
source: David B. Boardman, Geoffrey Greene, Vivek Khandelwal, and Aditya P. Mathur. LISTEN: A Tool to Investigate the Use of Sound for the Analysis of Program Behavior. In 19th International Computer Software and Applications Conference, Dallas, TX, 1995. IEEE.
http://www.cs.purdue.edu/homes/apm/listen/bubble.au
Example S18.5: CAITLIN
This is a sonification by the CAITLIN system of a Pascal CASE selection structure. The rising and falling melodic motifs signal the entry and exit of the selection, the cowbell steps through each of the candidate CASE instances, and the minor-key device denotes a failure to match any of the instances and the following of the default ELSE path.
media file S18.5
download: SHB-S18.5 (mp3, 243k)
source: Paul Vickers and James L. Alty. Musical program auralization: Empirical studies. ACM Trans. Appl. Percept., 2(4):477489, 2005. ISSN 1544-3558. doi: http://doi.acm.org/10.1145/1101530.1101546.
http://computing.unn.ac.uk/staff/cgpv1/caitlin/tutorial/tutorial2.htm
Example S18.6: slices
Berman and Gallagher sonified slices of Java programs rather than low-level constructs. This example sonifies a main method in a multi-method program. Tones represent source code statements with the higher pitches standing for statements in the slice and lower tones for statements outside the slice.
media file S18.6
download: SHB-S18.6 (mp3, 836k)
source: Lewis I. Berman and Keith B. Gallagher. Listening to Program Slices. In Tony Stockman, Louise Valgerður Nickerson, Christopher Frauenberger, Alistair D. N. Edwards, and Derek Brock, editors, ICAD 2006 – The 12th Meeting of the International Conference on Auditory Display, pages 172175, London, UK, 2023 June 2006. Department of Computer Science, Queen Mary, University of London, UK.
http://mysite.verizon.net/vze8b1hf/slice_sonif/slice_sonif.htm
Example S18.7: Peep
Peep followed a soundscape approach to the sonification of network activity in which network state information was gathered from multiple sources and was used to trigger the playback of pre-recorded natural sounds. This example of low network load creates a soundscape that represents the continuous and changing state of the network being monitored.
media file S18.7
download: SHB-S18.7 (mp3, 1.3M)
source: Michael Gilfix and Alva L. Couch. Peep (The Network Auralizer): Monitoring Your Network With Sound. In 14th System Administration Conference (LISA 2000), pages 109117, New Orleans, Louisiana, USA, 3–8 December 2000. The USENIX Association.
http://peep.sourceforge.net/intro.html
Example S18.8: WebMelody
Barra et al. sonified web server workload by mixing the sonifications with a user-selected piece of music. The sonification approach was based partly upon ideas of the Futurist movement which offered “an aesthetic connotation to sounds that border between music and background noises”.
media file S18.8
download: SHB-S18.8 (mp3, 940k)
source: Maria Barra, Tania Cillo, Antonio De Santis, Umberto F. Petrillo, Alberto Negro, and Vittorio Scarano. Multimodal Monitoring of Web Servers. IEEE Multimedia, 9(3):3241, 2002.
http://isis.dia.unisa.it/projects/SONIFICATION/sonifsite.html