Chapter 10: Laboratory Methods for Experimental Sonification

by Till Bovermann, Julian Rohrhuber and Alberto de Campo

Description

This chapter elaborates on sonification as an experimental method by argumenting that sonification methods need to incrementally merge into the specific cultures of research, including learning, drafting, handling of complexity, and communicatiing within and between multiple communities. The place where such a convergence may be found may be called a sonification laboratory.

Download Chapter

Download the chapter: TheSonificationHandbook-chapter10 (PDF, 1.1M)

Media Examples

Example S10.1: Pulse
Create a synth definition named “pulse”.


media file S10.1
download: SHB-S10.1 (mp3, 86k)
source: created by the authors, SuperCollider code provided below:

SynthDef(\pulse, { // create a synth definition named “pulse”
 |freq = 440, amp = 0.1| // controls that can be set at runtime
 Out.ar( // create an outlet for the sound
 0,     // on channel 0 (left)
 Pulse.ar( // play a pulsing signal
 freq // with the given frequency
 ) * amp // multiply it by the amp factor to determine its volume
 );
}).add; // add it to the pool of SynthDefs

x = Synth(\pulse);

x.set(\freq, 936.236);  // set the frequency of the Synth

x.free;

Example S10.2: Simple Map
Continuous mapping.


media file S10.2
download: SHB-S10.2 (mp3, 29k)
source: created by the authors, SuperCollider code provided below:

a = [ 191.73, 378.39, 649.01, 424.49, 883.94, 237.32, 677.15, 812.15 ];

Task {
 // instantiate synth
 x = Synth(\pulse, [\freq, 20, \amp, 0]);
 0.1.wait;

 x.set(\amp, 0.1);     // turn up volume
 // step through the array
 a.do{|item| // go through each item in array a
 // set freq to current value
 x.set(\freq, item);

 // wait 0.1 seconds
 0.1.wait;
 };

 // remove synth
 x.free;
}.play;

Example S10.3: Grain Map
Discrete mapping


media file S10.3
download: SHB-S10.3 (mp3, 29k)
source: created by the authors, sc code provided below

SynthDef(\sinegrain, {
 |out = 0, attack = 0.01, decay = 0.01, freq, pan = 0, amp = 0.5|

 var sound, env;

 // an amplitude envelope with fixed duration
 env = EnvGen.ar(Env.perc(attack, decay), doneAction: 2);

 // the underlying sound
 sound = FSinOsc.ar(freq);

 // use the envelope to control sound amplitude:
 sound = sound * (env * amp);

 // add stereo panning
 sound = Pan2.ar(sound, pan);

 // write to output bus
 Out.ar(out, sound)
}).add;

Synth.grain(\sinegrain, [\freq, 4040, \pan, 1.0.rand2]);

Task {
 // step through the array
 a.do{|item|
 // create synth with freq parameter set to current value
 // and set decay parameter to slightly overlap with next grain
 Synth.grain(\sinegrain, [\freq, item, \attack, 0.001, \decay, 0.2]);

 0.1.wait; // wait 0.1 seconds between grain onsets
 };
}.play;

Example S10.4: Buf Map
Buffer-based mapping.


media file S10.4
download: SHB-S10.4 (mp3, 207k)
source: created by the authors, SuperCollider code provided below:

b = Buffer.loadCollection(
 server: s,
 collection: a,
 numChannels: 1,
 action: {"load completed".inform}
 );

 SynthDef(\bufferSon, {|out = 0, buf = 0, rate = 1, t_trig = 1, amp = 0.5|
 var value, synthesis;

 value = PlayBuf.ar(
 numChannels: 1,
 bufnum: buf,
 rate: rate/SampleRate.ir,
 trigger: t_trig,
 loop: 0
 );

 synthesis = Saw.ar(value);

 // write to outbus
 Out.ar(out, synthesis * amp);
 }).add;

 x = Synth(\bufferSon, [\buf, b])

 x.set(\rate, 5); // set rate in samples per second
 x.set(\t_trig, 1);  // start from beginning
 x.free;             // free the synthesis process

Example S10.5: Audification
Audification.


media file S10.5
download: SHB-S10.5 (mp3, 1.6M)
source: created by the authors, SuperCollider code provided below:

SynthDef(\bufferAud, {|out = 0, buf = 0, rate = 1, t_trig = 1, amp = 0.5|

 var synthesis = PlayBuf.ar(
 numChannels: 1,
 bufnum: buf,
 rate: rate/SampleRate.ir,
 trigger: t_trig,
 loop: 0
 );

 // write to output bus
 Out.ar(out, synthesis * amp)
}).add;

a = {|i|cos(i**(sin(0.0175*i*i)))}!10000;
a.plot2; // show a graphical representation;

b = Buffer.loadCollection(
server: s,
collection: a,
numChannels: 1,
action: {"load completed".inform}
);

// create synth
x = Synth(\bufferAud, [\buf, b, \rate, 44100]);

x.set(\t_trig, 1);             // restart
x.set(\rate, 200);             // adjust rate
x.set(\t_trig, 1, \rate, 400); // restart with adjusted rate
x.set(\t_trig, 1, \rate, 1500);

x.free;

Example S10.6: Pattern
Discrete mapping based on patterns.


media file S10.6
download: SHB-S10.6 (mp3, 131k)
source: created by the authors, SuperCollider code provided below:

a = [ 191.73, 378.39, 649.01, 424.49, 883.94, 237.32, 677.15, 812.15 ];
Pbind(
 \instrument, \sinegrain,
 \freq, Pseq( a ),  // a sequence  of the dataset a
 \attack, 0.001,    // and fixed values as desired
 \decay, 0.2,       // for the other parameters
 \dur, 0.1
).play

Example S10.7: Pattern 2
Variation and extension of Pattern.


media file S10.7
download: SHB-S10.7 (mp3, 127k)
source: created by the authors, SuperCollider code provided below:

 a = [
 [ 161.58, 395.14 ], [ 975.38, 918.96 ], [ 381.84, 293.27 ],
 [ 179.11, 146.75 ], [ 697.64, 439.80 ], [ 202.50, 571.75 ],
 [ 361.50, 985.79 ], [ 550.85, 767.34 ], [ 706.91, 901.56 ],
 ]
 Pbind(
 \instrument, \sinegrain,
 \freq, Pseq( a ),  // a sequence  of the dataset a
 \attack, 0.001,    // and fixed values as desired
 \decay, 0.2,       // for the other parameters
 \dur, 0.1
 ).play

Example S10.8: Pattern 3
variation and extension of Pattern.


media file S10.8
download: SHB-S10.8 (mp3, 131k)
source: created by the authors, SuperCollider code provided below:

Pbind(
 \instrument, \sinegrain,
 \freq, Pseq( a ),  // a sequence  of the data (a)
 \attack, 0.001,
 \decay, 0.2,
 \pan, [-1, 1],  // pan first channel to left output, second to right
 \dur, 0.1
 ).play

Example S10.9: Order 1
Variations in order, see chapter for details.


media file S10.9
download: SHB-S10.9 (mp3, 283k)
source: created by the authors, SuperCollider code provided below:

a = [
 [ 0.97, 0.05, -0.22, 0.19, 0.53, -0.21, 0.54, 0.1, -0.35, 0.04 ],
 [ -0.07, 0.19,  0.67, 0.05, -0.91, 0.1,  -0.8, -0.21, 1, -0.17 ],
 [ 0.67, -0.05, -0.07, -0.05, 0.97, -0.65, -0.21, -0.8, 0.79, 0.75 ]
 ]; 

 SynthDef(\x, { |freq = 440, amp = 0.1, sustain = 1.0, out = 0|
 var sound = SinOsc.ar(freq);
 var env = EnvGen.kr(Env.perc(0.01, sustain, amp), doneAction: 2);
 Out.ar(out, sound * env);
 }).add;

 // define a mapping from number value to frequency:
 f = { |x| x.linexp(-1, 1, 250, 1000) };
 Task {
 var line = a[0]; // first line of data
 line.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.3.wait;
 }
 }.play;

Example S10.10: Order 2
Variations in order, see chapter for details.


media file S10.10
download: SHB-S10.10 (mp3, 135k)
source: created by the authors, SuperCollider code provided below:

Task {
 a.do { |line|
 line.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;
 }
 };
 0.3.wait;
 }.play;

Example S10.11: Order 3
Variations in order, see chapter for details.


media file S10.11
download: SHB-S10.11 (mp3, 221k)
source: created by the authors, SuperCollider code provided below:

Task {
 a.do { |line|
 line.copy.sort.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;
 };
 0.3.wait;
 }
 }.play;

Example S10.12: Order 4
Variations in order, see chapter for details.


media file S10.12
download: SHB-S10.12 (mp3, 53k)
source: created by the authors, SuperCollider code provided below:

Task {
 a.do { |line|
 line.do { |val|
 (instrument: \x, freq: f.value(val)).play; // no wait time here
 }
 };
 0.3.wait;
 }.play;

Example S10.13: Order 5
Variations in order, see chapter for details.


media file S10.13
download: SHB-S10.13 (mp3, 242k)
source: created by the authors, SuperCollider code provided below:

Task {
 var cols = a.flop; // swap rows <-> columns
 cols.do { |col|
 col.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;     // comment out for 3-note chords
 };
 0.3.wait;
 };
 }.play;

Example S10.14: Order 6
Variations in order, see chapter for details.


media file S10.14
download: SHB-S10.14 (mp3, 139k)
source: created by the authors, SuperCollider code provided below:

Task {
 var all = a.flat.sort;
 all.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;
 };
 }.play;

Example S10.15: mapping 1
Mapping strategies, see chapter for details.


media file S10.15
download: SHB-S10.15 (mp3, 373k)
source: created by the authors, SuperCollider code provided below:

// alternate the two close values
 Task { loop {
 [0.53, 0.54].do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;
 }
 } }.play;

 // then switch between different mappings:
 f = { |x| x.linexp(-1, 1, 250, 1000) }; // mapping as it was
 f = { |x| x.linexp(-1, 1, 500, 1000) };  // narrower
 f = { |x| x.linexp(-1, 1, 50, 10000) }; // much wider

Example S10.16: mapping 2
Mapping strategies, see chapter for details.


media file S10.16
download: SHB-S10.16 (mp3, 594k)
source: created by the authors, SuperCollider code provided below:

// run entire dataset with new mapping:
 Task {
 a.do { |line|
 line.do { |val|
 (instrument: \x, freq: f.value(val)).play;
 0.1.wait;
 }
 };
 0.3.wait;
 }.play;

Example S10.17: mapping 3
Mapping strategies, see chapter for details.


media file S10.17
download: SHB-S10.17 (mp3, 127k)
source: created by the authors, SuperCollider code provided below:

SynthDef(\x, { |freq = 440, amp = 0.1, sustain = 1.0, out = 0|
 var sound = SinOsc.ar(freq);
 var ampcomp = AmpComp.kr(freq.max(50));  // compensation factor
 var env = EnvGen.kr(Env.perc(0.01, sustain, amp), doneAction: 2);
 Out.ar(out, sound * ampcomp * env);
 }).add;

 Task {
 a.do { |line|
 line.do { |val|
 (instrument: \x, freq: f.value(val), sustain: 0.3).play;
 0.1.wait;
 };
 };
 0.3.wait;
 }.play;

Example S10.18: mapping 4
Mapping strategies, see chapter for details.


media file S10.18
download: SHB-S10.18 (mp3, 106k)
source: created by the authors, SuperCollider code provided below:

Task {
 var cols = a.flop; // swap rows <-> columns
 cols.do { |vals|
 var freq = vals[0].linexp(-1, 1, 300, 1000);
 var sustain = vals[1].linexp(-1, 1, 0.1, 1.0);
 var amp = vals[2].linexp(-1, 1, 0.03, 0.3); 

 (instrument: \x,
 freq: freq,
 sustain: sustain,
 amp: amp
 ).play;
 0.2.wait;
 };
 }.play;

Example S10.19: mapping 5
Mapping strategies, see chapter for details.


media file S10.19
download: SHB-S10.19 (mp3, 78k)
source: created by the authors, SuperCollider code provided below:

SynthDef(\xmod, { |freq = 440, modfreq = 440, moddepth = 0,
 amp = 0.1, sustain = 1.0, out = 0|
 var mod = SinOsc.ar(modfreq) * moddepth;
 var sound = SinOsc.ar(freq, mod);
 var env = EnvGen.kr(Env.perc(0.01, sustain, amp), doneAction: 2);
 Out.ar(out, sound * env);
 }).add;

 (
 Task {
 var cols = a.flop; // swap rows <-> columns
 cols.do { |vals|
 var freq = vals[0].linexp(-1, 1, 250, 1000);
 var modfreq = vals[1].linexp(-1, 1, 250, 1000);
 var moddepth = vals[2].linexp(-1, 1, 0.1, 4);
 (instrument: \xmod,
 modfreq: modfreq,
 moddepth: moddepth,
 freq: freq,
 sustain: 0.3,
 amp: 0.1
 ).postln.play;
 0.2.wait;
 };
 }.play;
 )

Example S10.20: Tree 1
Tree sonification. See chapter for details.


media file S10.20
download: SHB-S10.20 (mp3, 205k)
source: created by the authors, SuperCollider code provided below:

q = ();

 SynthDef(\x, { |freq = 440, amp = 0.1, sustain = 1.0, out = 0|
 var signal = SinOsc.ar(freq);
 var env = EnvGen.kr(Env.perc(0.01, sustain, amp), doneAction: 2);
 Out.ar(out, signal * env);
 }).add;

 (
 q.graph = [
 [0, 0, 0, 1, 0],
 [1, 0, 1, 0, 1],
 [0, 0, 1, 0, 0],
 [0, 0, 1, 0, 1],
 [0, 0, 1, 1, 0]
 ];
 // arbitrary set of pitches to label nodes:
 q.nodeNotes = (0..4) * 2.4; // equal tempered pentatonic

 )

 (
 Task {
 // iterating over all nodes (order defaults to listing order)
 loop {
 q.graph.do { |arrows, i|
 var basenote = q.nodeNotes[i];
 // find the indices of the connected nodes:
 var indices = arrows.collect { |x, i| if(x > 0,  i, nil) };
 // keep only the connected indices (remove nils)
 var connectedIndices = indices.select { |x| x.notNil };
 // look up their pitches/note values
 var connectedNotes = q.nodeNotes[connectedIndices];
 (instrument: \x, note: basenote).play;
 0.15.wait;

 (instrument: \x, note: connectedNotes).play;
 0.45.wait;
 };
 0.5.wait;
 };
 }.play;
 )

Example S10.21: Tree 2
Tree sonification. See chapter for details.


media file S10.21
download: SHB-S10.21 (mp3, 201k)
source: created by the authors, SuperCollider code provided below:

Task {
 var playAndChoose = { |nodeIndex = 0|
 var indices = q.graph[nodeIndex].collect { |x, i| if(x > 0,  i, nil) };
 var connectedIndices = indices.select { |x| x.notNil };

 var basenote = q.nodeNotes[nodeIndex].postln;
 var connectedNotes = q.nodeNotes[connectedIndices];

 (instrument: \x, note: basenote).play;
 0.15.wait;

 (instrument: \x, note: connectedNotes).play;
 0.3.wait;
 // pick one connection and follow it
 playAndChoose.value(connectedIndices.choose);
 };
 playAndChoose.value(q.size.rand); // start with a random node
 }.play;

Example S10.22: Algo 1
Sonifications of algorithms. See chapter for details.


media file S10.22
download: SHB-S10.22 (mp3, 287k)
source: created by the authors, SuperCollider code provided below:

(
 f = { |a, b|
 var t;
 while {
 b != 0
 } {
 t = b;
 b = a mod: b;
 a = t;
 };
 a
 };

 g = { |i| i * 100 }; // define a mapping from  natural numbers to frequencies.
 )

 SynthDef(\x, { |freq = 440, amp = 0.1, sustain = 1.0, out = 0|
 var signal = SinOsc.ar(freq) * AmpComp.ir(freq.max(50));
 var env = EnvGen.kr(Env.perc(0.01, sustain, amp), doneAction: 2);
 Out.ar(out, signal * env);
 }).add;

 Task {
 var n = 64;
 var a = { rrand (1, 100) } ! n; // a set n random numbers <= 100
 var b = { rrand (1, 100) } ! n; // and a second dataset.
 n.do { |i|
 var x = a[i], y = b[i];
 var gcd = f.value(x, y);
 var nums = [x, y, gcd].postln; // two operands and the result
 var freqs = g.value(nums);    // mapped to 3 freqs ...
 // in a chord of a sine grains
 (instrument: \x, freq: freqs).play;
 0.1.wait;
 }

 }.play

Example S10.23: Algo 2
Sonifications of algorithms. See chapter for details.


media file S10.23
download: SHB-S10.23 (mp3, 442k)
source: created by the authors, SuperCollider code provided below:

f = { |a, b, func|
 var t;
 while {
 b != 0
 } {        // return values before b can become 0
 func.value(a, b, t);
 t = b;
 b = a mod: b;
 a = t;
 };
 };

 // procedural sonification of the Euclidean algorithm
 Task {
 var n = 64;
 var a = { rrand (1, 100) } ! n; // n random numbers <= 100.
 var b = { rrand (1, 100) } ! n; // and a second dataset.
 n.do { |i|
 f.value(a[i], b[i], { |a, b| // pass the
 var numbers = [a, b].postln;
 // a 2 note chord of sine grains
 (instrument: \x, freq: g.value(numbers)).play;
 0.1.wait; // halt briefly after each step
 });
 0.5.wait; // longer pause after each pair of operands
 }
 }.play;

Example S10.24: Op 1
Operator-based sonification. See chapter for details.


media file S10.24
download: SHB-S10.24 (mp3, 94k)
source: created by the authors, SuperCollider code provided below:

(
 SynthDef(\fall, { |h0 = 30, duration = 3, freqScale = 30|
 var y, law, integral, g, t, h, freq, phase, k;
 g = 9.81; // gravity constant
 t = Line.ar(0, duration, duration); // advancing time (sec)
 law = { |t| g * t.squared }; // Newtonian free fall
 integral = { |x| Integrator.ar(x) * SampleDur.ir };
 h = h0 - law.value(t); // changing height
 freq = (max(h, 0) * freqScale); // stop at bottom, scale
 phase = integral.(freq); // calculate sin phase
 y = sin(2pi * phase);
 // output sound - envelope frees synth when done
 Out.ar(0, y * Linen.kr(h > 0, releaseTime:0.1, doneAction:2));
 }).add;
 );

 Synth(\fall);

Example S10.25: Op 2
Operator-based sonification. See chapter for details.


media file S10.25
download: SHB-S10.25 (mp3, 111k)
source: created by the authors, SuperCollider code provided below:

(
 Ndef(\x, {
 var law, g = 9.81, angle;
 var pointsOfTouch;
 var ball, time, sound, grid;

 //  pointsOfTouch = [1, 2, 3, 5, 8, 13, 21, 34]; // a wrong estimate
 // typical measured points by Riess et al (multiples of 3.1 cm):
 pointsOfTouch = [1, 4, 9, 16.1, 25.4, 35.5, 48.5, 63.7] * 0.031; 

 angle = 1.9; // inclination of the plane in degrees
 law = { |t, gravity, angle|
 sin(angle / 360 * 2pi) * gravity * squared(t)
 };

 // linear "procession" of time:
 time = Line.ar(0, 60, 60);
 // distance of ball from origin is a function of time:
 ball = law.value(time, g, angle); 

 sound = pointsOfTouch.collect { |distance, i|
 var passedPoint = ball > distance; // 0.0 if false, 1.0 if true
 // HPZ2: only a change from 0.0 to 1.0 triggers
 var trigger = HPZ2.ar(passedPoint);
 // simulate the ball hitting each gut fret
 Klank.ar(
 `[
 {exprand(100, 500) }    ! 5,
 { 1.0.rand }            ! 5,
 { exprand(0.02, 0.04) } ! 5
 ],
 Decay2.ar(trigger, 0.001, 0.01, PinkNoise.ar(1))
 )
 };

 // distribute points of touch in the stereo field from left to right
 Splay.ar(sound) * 10
 // optionally, add an acoustic reference grid
 //  + Ringz.ar(HPZ2.ar(ball % (1/4)), 5000, 0.01);
 }).play
 )

Example S10.26: Op 3
Operator-based sonification. See chapter for details.


media file S10.26
download: SHB-S10.26 (mp3, 90k)
source: created by the authors, SuperCollider code provided below:

(
Ndef(\x, {
 var law, g = 9.81, angle;
 var pointsOfTouch;
 var ball, time, sound, grid;

//  pointsOfTouch = [1, 2, 3, 5, 8, 13, 21, 34]; // a wrong estimate
 // typical measured points by Riess et al (multiples of 3.1 cm):
 pointsOfTouch = [1, 4, 9, 16.1, 25.4, 35.5, 48.5, 63.7] * 0.031; 

 angle = 1.9; // inclination of the plane in degrees
 law = { |t, gravity, angle|
 sin(angle / 360 * 2pi) * gravity * squared(t)
 };

 // linear "procession" of time:
 time = Line.ar(0, 60, 60);
 // distance of ball from origin is a function of time:
 ball = law.value(time, g, angle); 

 sound = pointsOfTouch.collect { |distance, i|
 var passedPoint = ball > distance; // 0.0 if false, 1.0 if true
 // HPZ2: only a change from 0.0 to 1.0 triggers
 var trigger = HPZ2.ar(passedPoint);
 // using Galileo's father's music theory for tone intervals
 var freq = 1040 * ((17/18) ** i);
 // simple vibrating string model by comb filter.
 CombL.ar(trigger, 0.1, 1/freq, 1.0 + 0.3.rand2)
 };

 // distribute points of touch in the stereo field from left to right
 Splay.ar(sound) * 10
 // optionally, add an acoustic reference grid
//  + Ringz.ar(HPZ2.ar(ball % (1/4)), 5000, 0.01);
}).play
)