Updating SDL2 audio callback data in Rust
Introduction
The Rust programming language has a somewhat-deserved reputation for sometimes getting in your face and preventing you from doing whatever you’re doing. This is more often than not related to the fact, that the Rust compiler works really hard to prevent data races from happening, at compile time and tries to save you from shooting yourself in the foot. It may be frustrating at times, but boy is it worth it (in my opinion). With the introduction of non-lexical lifetimes in Rust 1.31 (Rust 2018 edition) and Rust 1.36 (Rust 2015) the compiler is even smarter and allows several classes of programs previously forbidden from compiling. If you need to modify data formally owned by someone else you still need to jump through some hoops and that’s what we’ll explore here. Since one of my current projects is a reimplementation of a Polish real-time strategy game Polanie written in Rust and using SDL 2 that’s what we’ll work with here – Rust and SDL2 (thanks to the Rust bindings provided by the Rust-SDL2 project).
Starting with Rust and SDL2 audio
Let’s start a new project:
~/projets% cargo new rust-sdl2-audio-callback-modify
Created binary (application) `rust-sdl2-audio-callback-modify` package
~/projects% cd rust-sdl2-audio-callback-modify
We’ll be using the SDL2 bindings mentioned above, so let’s add a dependency to Cargo.toml (the bundled feature makes is so that you don’t need to have SDL2 installed in your system – the latest SDL2 version will be downloaded and compiled for you):
# Cargo.toml
# (...)
[dependencies]
sdl2 = {version = "*", features = ["bundled"]}
Let’s modify the autogenerated program (src/main.rs) to initialize SDL2’s audio subsystem and actively play silence for a second:
use sdl2::audio::{AudioCallback, AudioSpecDesired};
use std::thread;
use std::time::Duration;
struct SimpleCallback;
impl AudioCallback for SimpleCallback {
type Channel = i16;
// This function is called whenever the audio subsystem wants more data to play
fn callback(&mut self, out: &mut [i16]) {
for value in out.iter_mut() {
*value = 0;
}
}
}
fn main() {
let sdl_context = sdl2::init().unwrap();
let audio_subsystem = sdl_context.audio().unwrap();
let desired_audio_spec = AudioSpecDesired {
freq: Some(44_100),
// Mono
channels: Some(1),
// Doesn't matter here, use the default value
samples: None,
};
let audio_device = audio_subsystem
.open_playback(None, &desired_audio_spec, |_spec| SimpleCallback {})
.unwrap();
// This starts the playback.
audio_device.resume();
thread::sleep(Duration::from_millis(1_000));
}
See the SDL_AudioSpec documentation for some details about the callback-based mechanism for providing the audio subsystem the raw audio data – in short it’s a pull-based model to get more data from you when it’s needed. Note that we have to actively fill the whole buffer because The callback must completely initialize the buffer; as of SDL 2.0, this buffer is not initialized before the callback is called. If there is nothing to play, the callback should fill the buffer with silence.
A digression on silence
See how in the code above we use 0 as silence:
fn callback(&mut self, out: &mut [i16]) {
for value in out.iter_mut() {
*value = 0;
}
}
0 is not always the right value though (I learned about it the hard way) – if your channel type is u8 then the value of silence is 128. Why? Let’s think of a sine wave represented in u8. It’ll oscillate between 0 and 255! With the channel type using 0 for silence is just as good as using 255 – not very. The “neutral” value is 128 – the middle of the range. This is reflected by the fact that the AudioSpec structure that SDL gives you when opening the audio device contains a calculated silence value that’s based on your channel type. The calculation is simple enough that you could remember to just use 0 except for channels of type u8:
// SDL 2.0.9, src/audio/SDL_audio.c
case AUDIO_U8:
spec->silence = 0x80;
break;
default:
spec->silence = 0x00;
break;
If you accidentally fill a u8 buffer with zeros when you want silence you’ll get nasty clicking.
Back to the task at hand
Given a sound file – how do we play it? With a WAV file it’s simple enough. We need to add it to the project, load and convert it to a format compatible with our audio spec:
let audio_device = audio_subsystem
.open_playback(None, &desired_audio_spec, |spec| {
let wav = AudioSpecWAV::load_wav("beep.wav").unwrap();
let converter = AudioCVT::new(
wav.format,
wav.channels,
wav.freq,
spec.format,
spec.channels,
spec.freq,
)
.unwrap();
let data = converter.convert(wav.buffer().to_vec());
SimpleCallback {}
})
.unwrap();
Then we’ll store the data in the callback structure (so that it can access it when SDL requests more data):
struct SimpleCallback {
buffer: Vec<u8>,
position: usize,
}
// (...)
let audio_device = audio_subsystem
.open_playback(None, &desired_audio_spec, |spec| {
let wav = AudioSpecWAV::load_wav("beep.wav").unwrap();
let converter = AudioCVT::new(
wav.format,
wav.channels,
wav.freq,
spec.format,
spec.channels,
spec.freq,
)
.unwrap();
let data = converter.convert(wav.buffer().to_vec());
// This is the modified fragment
SimpleCallback {
buffer: data,
position: 0,
}
})
.unwrap();
Then, finally, let’s modify the callback to feed the audio subsystem with the data. We’ll feed it as much data as we have (increasing the byte position) and after than we’ll play silence. Note that we need to convert between a vector of u8 (that’s how we read the WAV data) and i16 (the type of the channel). I’m assuming little-endian architecture, hence from_le_bytes:
fn callback(&mut self, out: &mut [i16]) {
for value in out.iter_mut() {
*value = if self.position < self.buffer.len() {
let sample = i16::from_le_bytes([
self.buffer[self.position],
self.buffer[self.position + 1],
]);
self.position += 2;
sample
} else {
0
}
}
}
Execute cargo run and enjoy a beep being played.
The problem
At some point you’ll want to modify the data the callback operates on to change the underlying buffer or update the position to, for example, play the same sound twice, with a thousand millisecond delay between its two playbacks. A naive approach to mutate the callback structure from two contexts will fail (not in the way you’d expect though, in this case – for backwards compatibility Rust 1.38 allows the code to compile, but it’ll exhibit undefined behavior and won’t work as expected):
let mut callback = SimpleCallback {
buffer: Vec::new(),
position: 0,
};
let audio_device = audio_subsystem
.open_playback(None, &desired_audio_spec, |spec| {
let wav = AudioSpecWAV::load_wav("beep.wav").unwrap();
let converter = AudioCVT::new(
wav.format,
wav.channels,
wav.freq,
spec.format,
spec.channels,
spec.freq,
)
.unwrap();
let data = converter.convert(wav.buffer().to_vec());
callback.buffer = data;
callback
})
.unwrap();
// This starts the playback.
audio_device.resume();
thread::sleep(Duration::from_millis(1_000));
// Play the sound once again, effectively
callback.position = 0;
thread::sleep(Duration::from_millis(1_000));
Build it:
~/projects/rust-sdl2-audio-callback-modify% cargo run
Compiling rust-sdl2-audio-callback-modify v0.1.0 (/Users/user/projects/rust-sdl2-audio-callback-modify)
warning[E0382]: assign to part of moved value: `callback`
--> src/main.rs:70:5
|
42 | let mut callback = SimpleCallback {
| ------------ move occurs because `callback` has type `SimpleCallback`, which does not implement the `Copy` trait
...
48 | .open_playback(None, &desired_audio_spec, |spec| {
| ------ value moved into closure here
...
60 | callback.buffer = data;
| -------- variable moved due to use in closure
...
70 | callback.position = 0;
| ^^^^^^^^^^^^^^^^^^^^^ value partially assigned here after move
|
= warning: this error has been downgraded to a warning for backwards compatibility with previous releases
= warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future
= note: for more information, try `rustc --explain E0729`
Finished dev [unoptimized + debuginfo] target(s) in 0.41s
Running `target/debug/rust-sdl2-audio-callback-modify`
Run it with cargo run – it only plays the sound once. Other approaches to share and mutate the data from multiple contexts won’t even be allowed to compile. How should we proceed? Do we need to hide the data behind a mutex? No – we don’t have to deal with mutexes directly. Rust-SDL2 provides a solution. The AudioDevice structure has a lock method. lock() returns an instance of AudioDeviceLockGuard which locks the audio subsystem, lets you access the callback through dereferencing and unlocks the audio subsystem when dropped. Using it is simple enough:
// We need to make audio_device mutable...
let mut audio_device = audio_subsystem
// (...)
thread::sleep(Duration::from_millis(1_000));
{
// ... so we can do this later:
let mut lock = audio_device.lock();
// lock dereferences to SimpleCallback so we can access SimpleCallback's attributes
// directly
lock.position = 0;
}
thread::sleep(Duration::from_millis(1_000));
Run it with cargo run – it plays the beep twice now, just as expected.
Summary
I hope this post sufficiently explains how to deal with updating the data of SDL2 audio callback in Rust. Only after implementing this in openpol I discovered SDL_QueueAudio – a push-based method to provide audio data added in SDL 2.0.4. I’m yet to use it.
You can find a self-contained repository containing code from this post here.