by Perry R. Cook
Description
This chapter covers most means for synthesizing sounds, with an emphasis on describing the parameters available for each technique, especially as they might be useful for data sonification. The techniques are covered in progression, from the least parametric (the fewest means of modifying the resulting sound from data or controllers), to the most parametric (most flexible for manipulation). Some examples are provided of using various synthesis techniques to sonify body position, desktop GUI interactions, stock data, etc.
Download Chapter
Download the chapter: TheSonificationHandbook-chapter9 (PDF, 3M)
Media Examples
All media examples were generated by the Author of this chapter, Perry R. Cook.
Example S9.1: Diphone Speech Singing Display
PCM Diphone Synthesis Example: Diphone Speech by concatenation of diphones, with pitch shift. Diphone “Singing” example using the same small diphone library. Interactive display of laptop tilt controls pitch and crossfade of two vowels.
media file S9.1
download: SHB-S9.1 (mp3, 827k)
source: Generated by the author of this chapter.
Example S9.2: Tilt Vowels
The tilt-controlled pitch and vowels example of syn-1 is also available as a Movie File.
media file S9.2
download: SHB-S9.2 (mp4, 836k)
source: Generated by the author of this chapter using ChucK language.
Example S9.3: Phase Vocoder
FFT Analysis/Resynthsis, Phase Vocoder Example: The phase vocoder allows manipulation of pitch, time, and spectrum independently.
media file S9.3
download: SHB-S9.3 (mp3, 436k)
source: Generated by the author of this chapter using pvoc (CARL) software.
Example S9.4: Modal Synthesis Sampler
Wood/Metal Modal Synthesis Example: Using only a 4-mode (resonant filter) model, varying the parameters of modal resonance frequencies, mode dampings, and excitation, allows for a wide variety of different sounds, from just one computationally efficient algorithm.
media file S9.4
download: SHB-S9.4 (mp3, 872k)
source: Generated by the author of this chapter using the Synthesis Toolkit in C++.
Example S9.5: LPC Cross Synth Example
LPC Example: LPC allows pitch scaling, time scaling, and other manipulations (similar to the phase vocoder, but very different in implementation). This example shows the “cross synthesis” of speech with a lion roaring. LPC analysis/coding was performed on both sounds, then the residue of the Lion was used as a source to drive the filters derived from the speech analysis.
media file S9.5
download: SHB-S9.5 (mp3, 238k)
source: Generated by the author of this chapter using custom LPC software in ANSI C.
Example S9.6: Body Formant Sonification
Parallel Formant Synthesis Example: Here a parallel formant model is controlled in real-time by a 6DOF (degree of freedom) two-hand tether device. The left hand controls voiced (pitched) vs. noise, and the pitch of the periodic source. The right hand controls the center frequencies of the three formant frequencies.
media file S9.6
download: SHB-S9.6 (mp3, 651k)
source: Generated by the author of this chapter using the ChucK language.
Example S9.7: Body Formant Sonification
The above S9.6 example of 6DOF (degree of freedom) controlled Parallel Formant Synthesis is also available as a Movie file.
media file S9.7
download: SHB-S9.7 (mp4, 1M)
source: Generated by the author of this chapter using the ChucK language.
Example S9.8: FM Synthesis Example
FM Example: Simple FM with C:M frequency ratio of 1.0 (both=300Hz), Index of modulation (deltaF/F) ranges from 0 to 7 in increments of 1. Then, with index=4, C:M is gradually changed from harmonic (1.0) to weakly inharmonic (1.1).
media file S9.8
download: SHB-S9.8 (mp3, 434k)
source: Generated by the author of this chapter using the ChucK language.
Example S9.9: Sonified Desktop
PHISEM Sonified Desktop/Scrolling: Moving the mouse over the desktop is sonified by the sound of sand(paper). Left-right cursor position pans sandpaper sound left and right. Headphones or stereo speakers recommended. Browser scrolling up and down is sonified by the pitch of a shaking tambourine model.
media file S9.9
download: SHB-S9.9 (mp4, 1.4M)
source: Generated by the author of this chapter using the ChucK language.
Example S9.10: Stock and BP Sonification
Marimba/String Stock example. Sonification of stock and other data. Stock price (normalized to $1 purchase on day 1 of each stock) is mapped to pitch and the daily trading volume to volume. Stock A (RHAT) is sonified by a plucked string in the left speaker (pitch=price, volume=volume) Stock B (MSFT) is sonified by a struck marimba in the right speaker (pitch=price, volume=volume). Stock holder’s diastolic blood pressure is mapped to the center frequency of filtered noise, panned center in the stereo image. Heaphones or stereo speakers recommended.
media file S9.10
download: SHB-S9.10 (mp3, 631k)
source: Generated by the author of this chapter using the ChucK language.