by Gregory Kramer
The book you’re holding, or perhaps reading on a screen, represents a sea change: the maturation of the field of Auditory Display (AD). It represents the aggregate work of a global community of inquiry as well as the labors of its individual editors and authors. Nineteen years ago—in 1992 and 1993—I was editing another book, one that would be published in 1994 as part of the Santa Fe Institute’s Studies in the Sciences of Complexity, Auditory Display: Sonification, Audification, and Auditory Interfaces. Although there had certainly been research papers that pre-dated it, this 1994 publication seemed to have the effect of catalyzing the field of auditory display research.
Up until the seminal 1992 conference—little more than a workshop with an outsized title, International Conference on Auditory Display—only scattered attention had been given to auditory interfaces generally, and nearly none to using sound as a means of conveying data. When I edited the conference proceedings into a book (with the feature, unusual for its time, of being sold with an audio CD included), and wrote an introduction that I hoped would provide some context and orienting theory for the field, the threshold of significance was modest. The vision, the fact of these unique papers, and a little weaving of them into a coherent whole was enough.
That is no longer the case. Nearly twenty years have passed since ICAD 92. A new generation of researchers has earned Ph.D.’s: researchers whose dissertation research has been in this field, advised by longtime participants in the global ICAD community. Technologies that support AD have matured. AD has been integrated into significant (read “funded” and “respectable”) research initiatives. Some forward thinking universities and research centers have established ongoing AD programs. And the great need to involve the entire human perceptual system in understanding complex data, monitoring processes, and providing effective interfaces has persisted and increased. The book that was needed twenty years ago is not the book needed now.
The Sonification Handbook fills the need for a new reference and workbook for the field, and does so with strength and elegance. I’ve watched as Thomas, Andy, and John have shepherded this project for several years. The job they had is very different from the one I had, but by no means easier. Finding strong contributions in 1990 often meant hunting, then cajoling, then arduous editing to make the individual papers clear and the whole project effective and coherent. Now, the field has many good people in it, and they can find each other easily (at the beginning of the 1990′s, the Web was still a “wow, look at that” experiment). With the bar so much higher, these editors have set high standards of quality and have helped authors who face the same time famine as everyone else to bring their chapters to fruition. Some of the papers included in the 1994 book were excellent; some were essentially conference papers, sketches of some possibility, because that’s what was available at the time. That book was both a reference source and documentation of the rise of a new field. Now there is a foundation of solid work to draw from and a body of literature to cite. In consequence, the present book is more fully and truly a reference handbook.
Just as compelling, there is a clear need for this book. When a field is first being defined, who’s to say that there is any need for that field—let alone for a book proffering both a body of work and the theoretical underpinnings for it. The current need includes the obvious demand for an updated, central reference source for the field. There is also a need for a book from which to teach, as well as a book to help one enter a field that is still fabulously interdisciplinary. And there is need for a volume that states the case for some of the pioneering work such as sonification and audification of complex data, advanced alarms, and non-traditional auditory interfaces. That we still call this work “pioneering” after twenty or thirty years of effort remains a notion worth investigating.
At ICAD conferences, and in many of the labs where AD research is undertaken, you’ll still find a community in process of defining itself. Is this human interface design, broadly speaking? Is it computer science? Psychology? Engineering? Even music? Old questions, but this multi-disciplinary field still faces them. And there are other now-classic challenges: when it comes to understanding data, vision still reigns as king. That the ears have vast advantages in contributing to understanding much temporally demanding or highly multi-dimension data has not yet turned the tide of funding in a significant way. There are commercial margins, too, with efforts progressing more in interfaces for the blind and less in the fields of medicine, financial data monitoring or analysis, and process control, long targets of experimental auditory displays. The cultural bias to view visually displayed data as more objective and trustworthy than what can be heard remains firmly established. Techniques to share and navigate data using sound will only become accepted gradually.
Perhaps the community of researchers that finds commonality and support at the ICAD conferences, as well as at other meetings involved with sound, such as ISon, Audio Mostly, and HAID, will have some contributions to make to understanding the human experience that are just now ready to blossom. New research shows that music activates a broad array of systems in the brain—a fact which, perhaps, contributes to its ubiquity and compelling force in all the world’s cultures. Might this hold a key to what is possible in well designed auditory displays? Likewise, advances in neuroscience point to complex interactions among auditory, visual, and haptic-tactile processing, suggesting that the omission from a design process of any sensory system will mean that the information and meanings derived, and the affective engagement invoked, will be decreased; everything from realism to user satisfaction, from dimensionality to ease of use, will suffer unacceptably.
I’ve been asked many times, “Where are things going in this field?” I have no idea! And that’s the beauty of it. Yes, AD suffers the curse of engaging so many other research areas that it struggles to find research funding, a departmental home in academia, and a clear sense of its own boundaries. The breadth that challenges also enriches. Every advance in auditory perception, sound and music computing, media technology, human interface design, and cognition opens up new possibilities in AD research. Where is it all leading? In this endeavor, we all must trust the emergent process.
When I began to put together the first ICAD conference in 1990, it took me a couple of years of following leads to find people currently doing, or recently involved in, any work in the field whatsoever. From the meager list I’d assembled, I then had to virtually beg people to attend the gathering, as if coming to Santa Fe, New Mexico, in the sunny, bright November of 1992 was insufficient motivation. In the end, thirty-six of us were there. Now, about 20 years later, a vibrant young field has emerged, with a global community of inquiry. The Sonification Handbook is a major step in this field’s maturation and will serve to unify, advance, and challenge the scientific community in important ways. It is impressive that its authors and editors have sacrificed the “brownie point” path of publishing for maximum academic career leverage, electing instead to publish this book as OpenAccess, freely available to anybody. It is an acknowledgement of this research community’s commitment to freely share information, enthusiasm, and ideas, while maintaining innovation, clarity, and scientific value. I trust that this book will be useful for students and newcomers to the field, and will serve those of us who have been deeply immersed in auditory displays all these years. It is certainly a rich resource. And yet—it’s always just beginning. The Sonification Handbook contributes needed traction for this journey.
Orcas Island, Washington