Ross Bencina writes about real-time audio programming, sound synthesis, and audio software libraries.
HN commenters call his real-time audio programming article spot-on and frequently cited, and recommend it for anyone writing reliable audio software.
I try and learn Rust by glueing all things together (MIDI, Open Sound Control, Audio I/O) into little commandline utilities (e.g. routers), it's mostly to keep myself happy; quick results keep me pushing. Whether it's a good idea to start with a plugin depends on your experience (e.g. writing such things in C or C++).
Active noise cancellation would be too involved for a first attempt (at least for me). Here's an unfinished project: https://github.com/materoy/noice
For regular denoising there's these (haven't tried them):
https://github.com/jneem/nnnoiseless
https://github.com/bennetthardwick/rust-obs-plugins/blob/master/plugins/rnnoise-denoiser-filter.
Here's an aggregate for sound related things: https://rust.audio/
Overviews/libraries:
https://github.com/kfrncs/awesome-rust-audio
https://crates.io/keywords/audio
• Some videos:
- Rust for audio developers:
https://www.youtube.com/watch?v=Yom9E-67bdI
- The nannou framework:
https://www.youtube.com/watch?v=JPFv3adyLB4
• Here's a real good article by Ross Bencina that goes into the intricacies of realtime audio, don't allocate/free in the hot parts of your code !
http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
Other things to check out:
fundsp - a compact library with DSL for typical dsp things
https://github.com/SamiPerttu/fundsp
nih-plug - a plugin framework
https://github.com/robbert-vdh/nih-plug/
Nannou
a creative coding library that glues the needed parts together in a…
Running Firefox on Windows 10 x64, on a somewhat under-specced machine that tends to run low on RAM.
I'm getting highly inconsistent note timings, with intermittent full 1-second pauses with no new notes being triggered. (FWIW, I also saw something similar in https://news.ycombinator.com/item?id=26870666 , and am getting choppy audio in Zoom that disappears as soon as I enable profiling...)
Is it my browser's fault for having the JS event loop stall, or the page's fault for driving audio off the page's event loop? Or both? (IDK how to fix Firefox, other than increasing the content processes above 3, which will reduce cross-tab lag but possibly increase RAM, or to reduce my extensions or find the one at fault.)
On a related note, how critical is it to follow the "audio on a real-time thread" dogma (http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing)? I've heard that not using Electron, writing real-time code, avoiding mutexes and allocations, mlock to avoid swapping, tuning OSes and picking processors and audio interfaces based on its low-latency properties, etc. is important for live performances, but not necessary for MIDI instrument input for (casual home users? studio work?).
I don't mean to devalue the advice here. I think it's spot on, and I unreservedly recommend this article to folks who want to learn about writing reliable audio software.
I think in essence I'm repeating the comments of Justin from Cockos, which you summarize [1]:
> It is basically saying that you can reduce the risk of priority inversion to the point where the probability is too low to worry about.
In that comment you also say:
> 100% certainty can’t be guaranteed without a hard real-time OS. However 5ms is now considered a relatively high latency setting in pro/prosumer audio circles
Which I interpret as acknowledging that we're already forced into the regime of establishing an acceptable level of risk.
My point is that I would love to see more data on the actual latency distributions we can expect, so that we can make more informed risk assessments. For example, I know that not all `std::atomic` operations are lock-free, but when the critical section is so small, is it really a problem in practice? I want histograms!
[1]: http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing#comment-87
Two books I loved during my digital signal processing studies (especially in the audio field):
- "Introduction to Computer Music"[0] by Nick Collins: very focused on audio programming for musical applications but you can easily extrapolate for other domains. Lots of pseudo-code algorithms are provided.
- "The Scientist and Engineer's Guide to Digital Signal Processing"[1] by Steven W. Smith: the bread and butter for everything concept related to DSP. Everything is explained in a crystal-clear fashion, lots of sample code to get you started with digital filters and analysis (although written in BASIC but easily translatable to C/C++ or even Python).
Also there is this one article "Real-time audio programming 101: time waits for nothing"[2] by Ross Bencina (creator of AudioMulch, Portaudio, etc.) about the basics concepts of real-time audio programming
[0] https://www.wiley.com/en-gb/Introduction+to+Computer+Music-p-9780470714553
[1] https://www.dspguide.com/
[2] http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
Personally, I started by writing externals for Pure Data, then started to contribute to the core. Later I took the same path for SuperCollider.
You may start with simple audio plugins. Have a look at JUCE (https://juce.com/)!
Realtime audio programming has some rather strict requirements that you don't have in most other software. Check out this classic article: http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
In WSM it's possible to create models that call external function in DLLs. I've been using a modelica wrapper to the oscpack library (http://www.rossbencina.com/code/oscpack) that allows sending and receiving simple OSC messages. This makes it possible to communicate with music applications that implement OSC.
I've been using it to send input signals from an iPhone using TouchOsc.