Mecheye

Audio worklets are a step into the right direction for streaming audio, but for a simple audio-streaming API you don't need the complex node-based architecture of WebAudio, you just need a JS callback which writes samples into an ArrayBuffer.

With audio worklets this callbacks runs in a separate audio thread, and with the (deprecated) ScriptProcessNode this callback runs on the main thread.

E.g. a "good" web audio API replacement would only offer a callback that runs in a separate audio thread plus a convenience function call which allows to push small sample-packets from the main thread to the audio thread (at the cost of increased latency to avoid starving) - this push-function would basically be the replacement for ScriptProcessorNode.

In general, see here for a pretty good overview why WebAudio as a whole is a badly designed API: https://blog.mecheye.net/2017/09/i-dont-know-who-the-web-audio-api-is-designed-for/

TL;DR: WebAudio's original design requires a lot of complexity and implementation effort for use cases that are not relevant to most of its users - and all that effort could be used instead to implement a much smaller and focused web audio API that covers actually relevant use cases.

Specifically for audio worklets: those mainly make sense when the entire audio stream generation can happen on the audio thread.

But if you need to generate audio on the main thread (such as in emulators: https://floooh.github.io/tiny8bit/), unless you want to run the entire…

I would say the most likely reason is that 2D graphics is just plain hard. Jasper's classic essay[1] goes into more detail.

That said, I'd really love to see more stuff along the lines of what you describe. I think Vello could render, for example, something that visually looks like Mini Metro or Mini Motorways. I also of course would be a massive fan of using Euler spirals - the only curve that's truly, really simpler is quadratic Béziers, and making those look good in your application would be hard. I'm about to do a lot of Euler spiral math on GPU as part of the stroke rework, which you may find interesting.

Strokes are easier for GPUs than fills, because you don't have to worry about the winding number issues (non-convex paths, paths with holes, etc). There are two recent papers, Polar Stroking[2] and Converting stroked primitives to filled primitives[3].

Also check out Freya Holmér's excellent Shapes library[4] (easily integrated with Unity).

Tesselating curves via shaders is the future (through mesh shaders in particular), and I've got a colleague working on that, but portable infrastructure seems to me just as thorny a problem as compute. I am very biased, but if it were me I'd try to render the 2D stuff with a 2D renderer like Vello.

Best of luck with your project, and I'd be interested in following the progress. Feel free to stop by the Zulip[5] and ask such questions there, as well.

[1]:…

You know what I meant by "infinite resolution". Today's monitors are nowhere close to retina (human eye resolution) and even further from being diffraction limited (the color frequency stuff you're talking about)

> If the monitors resolution and the font resolution were identical

Fonts are typically infinite resolution because they are small programs that you execute. Also see https://blog.mecheye.net/2012/12/bytecode/

> The brain processes sound logarithmically. Is the same true for vision?

If you're talking about brightness, it's a cube-root curve: https://en.wikipedia.org/wiki/Lightness#Relationship_to_value_and_relative_luminance

> I watch movies at 24 frames per second, and yet, I'm usually incapable of perceiving this

Film projectors flash the same image multiple times because humans can easily notice 24fps. The lamp flickers and that's why movies are called "flicks". Good monitors flash each frame for only 1ms for the same reason: https://blurbusters.com/faq/motion-blur-reduction/

Try the UFO test: https://www.testufo.com/. If you genuinely can't see the difference between 30/60/120 Hz, you should get an eye exam because that's not normal.

I wouldn't count on any other browser to be anywhere onboard with that. Chrome is known for implementing things in a quick and dirty way with complete disregard for input.

Also, Fugu will definitely no bring a signinficant amount of native APIs to the web. Here's the full list [1]

For example, the post I'm replying to mentioned Ableton. Well "I don’t know who the Web Audio API is designed for" [2] Oh, and then there's no guarantee Chrome will not break it the way it broke automatic audio previously [3]

The web is not the long-term future for any foreseeable future

[1] https://bugs.chromium.org/p/chromium/issues/list?can=2&q=proj-fugu&sort=pri&colspec=ID+Pri+M+Stars+ReleaseBlock+Component+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=ids

[2] https://blog.mecheye.net/2017/09/i-dont-know-who-the-web-audio-api-is-designed-for/

[3] https://twitter.com/mcclure111/status/993627449991688193?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E993627449991688193&ref_url=https%3A%2F%2Fwww.pcgamer.com%2Fa-google-chrome-update-breaks-the-audio-in-numerous-web-based-games%2F

This is exciting because it addresses one of the main complaints in the recent blog post http://blog.mecheye.net/2017/09/i-dont-know-who-the-web-audio-api-is-designed-for/, which is that there's no way in WebAudio to just fill a buffer with numbers (samples) and have it play them back in real time (or to do the same, but processing an input buffer).

AudioWorklet (https://webaudio.github.io/web-audio-api/#AudioWorklet) allows you to create "processor" nodes, which have a "process" method which is called once per buffer, with any audio input to the node passed in as a parameter, and an output buffer to fill.

This should allow much more interesting and professional level audio generation and effects using WebAudio, and much easier porting of existing audio algorithms (as this is how most audio code, e.g. VST/AudioUnit plugins work), though I've not looked at the spec in detail yet or played with it, so the devil as always is in the details!.

Reminds me of the audio API mess. If you want to stream arbitrary sample data to an output you can either use ScriptProcessorNode (simple API, well supported but deprecated) or AudioWorkletNode (over-engineered API, largely unsupported and experimental).

The stall in that case seems caused by the overly ambitious design of the Web Audio API altogether. It's utterly useless for audio processing or synthesis except when using the aforementioned APIs because it mandates a bunch of underspecified high level concepts. Just let me fill a buffer with floats for the speakers and read a buffer of floats for the mic. That's literally all I need, and literally the only way you can build, say, a compressor and have full control of its characteristics.

When researching the best option I found this, so I'm not alone in thinking this: https://blog.mecheye.net/2017/09/i-dont-know-who-the-web-audio-api-is-designed-for/