Magenta

TL;DR: If drumbot.glitch.me doesn't work for you, try running it locally [2]. It's surprisingly effortless!

First attempt at trying this out: 502 Bad Gateway.

Second attempt: I thought I didn't want to mess with Node, so I downloaded Magenta Studio [0] instead. When trying to load the Max patch, my potato promptly lost its will to Live. (Ba-dum tss [1].) Maybe loading a 900MB .amxd when you only have 1GB of free RAM isn't a great idea? Anyway, I'm not keen on risking another reboot today, so I'm putting this off for now.

Third attempt: Localhosting DrumBot [2]. Five minutes ago, I didn't have Node installed (I didn't even recall ever touching Node directly at all), but now (npm install … added 247 packages from 284 contributors … found 4 moderate severity vulnerabilities … npm start) the tensors are flowing already! Whoa, Node is so easy, any idiot can use it — even musicians ;)

I peck in a simple melody:

Fly me to the moon
Let me play Fly me to

Oops, this started looping at an unexpected point. I guess I should have stuck to a 4/4 time signature? DrumBot is equally confused and lethargically bangs out wonky shit like it's attempting to cure carbon monoxide poisoning with ketamine. Meet my new band: Tourette's Horse Tranquilizer.

I start over, this time with a proper MIDI keyboard plugged in. Initially this fails: MIDI Off messages trigger notes in the same way as MIDI On messages. This is easily fixed by routing the MIDI through a DAW. (Why? No idea.) I play a…

Are you familiar with Magenta? [0]
It's not real-time, but definitely an important area of research.

Also check out Dan Tepfer [1] who is doing amazing work with an algorithmic approach to reactive live performances, with great call and response tactics.

I myself am slowly prototyping a fully artificial AI band which can be orchestrated using very high-level musical ideas and a big helping of intelligent randomization and algorithms based on music theory.

I've been prototyping in Andrew Sorensen's Extempore [2] and have laid much of the groundwork like melody/harmony/rhythm generation as well as basic modulation of these elements to create longer musical structures which utilize motifs in multiple ways

Currently it is a matter of shedding pre-computed tables of "nice" sounding progressions or purely random progressions, and creating a more fundamental approach which can derive the appropriate progressions from the given user parameters. I am also expanding the program's ability to generate aesthetically pleasing and unique singular motifs which drive these algorithmic compositions.

I have a band leader / conductor module which provides cues and other synchronized data, which even without advanced motif generation and modulation still allows for things not currently possible in any non-code musical production software such as global dyamics, timing changes, progression sharing, (eventually) directing impromptu solos, etc.

Reach out via email if you'd like to discuss…

Glad to see people posting links to music modeling projects - it's a huge space, and a ton of fun to actively work on or just mess with if you like music and code. Even simple markov models can get someplace with a bit of tweaking [0][1].

If you like this, you might like some more recent work from Anna huang et. al. at Magenta, using a modified Transformer network [2][3].

Some other relevant reading includes DeepBach [4][5], and outside the domain of Bach alone, the venerable folkRNN [6]. There's a huge chain of work I posted a while back here [7], and Kyle McDonald wrote a really great review piece here [8].

Modeling music in latent spaces has also seen a number of recent publications [9][10][11], and having smooth latent spaces for generation seems (to me) to be a very natural interface for manipulating things, which has a natural analogue in "real" composition, cool video at [12].

There have also been some really fun online demos for interacting with these types of systems [13]

There's a lot of interesting work going on in music modeling, hopefully leading to new and interesting musical tools and toys. Interactivity (often via clever conditioning) can really change the way you approach these types of problems [14].

[0] https://github.com/kastnerkyle/magenta/tree/my_cleanup_branch/exp/markov_lm

[1] https://www.youtube.com/watch?v=Ay8mHW0-NW8&list=PLRMa_gJ8vx8mC9VvtXE7eGamGdf3MxzV4

[2] https://twitter.com/kastnerkyle/status/1018617315196309504

[3]…

PLL is an interesting option, never thought of that, that is worth pursuing, thank you. And like the other comment already wrote, state of the art.

The project is automatic music transcription, the 'gold standard' that I've found so far is Magenta based 'Onsets and Frames' (https://magenta.tensorflow.org/onsets-frames ), which scores pretty good on a large benchmark.

I'm still a ways away from being their score but there is real chance that it will happen. The tensorflow based software is super slow, many orders of magnitude slower than realtime would require and I have a bunch of applications in mind where slower than realtime is not an option.

Here are the two side-by-side across the same testset (first mine):

// di 24 dec 2019 16:49:29 CET overall total 17482 correct 12604 fp(fantasized) 2723 fn(missed) 4878 f_measure 76.833 precision 82.234 recall 72.097 accuracy 62.381

vs (Magenta O&F):

// wo 25 dec 2019 20:09:51 CET overall total 17482 correct 13865 fp(fantasized) 2908 fn(missed) 3617 f_measure 80.952 precision 82.663 recall 79.31 accuracy 67.999

Clearly the O&F software has the edge in detection but it is close enough that I have some hope of beating that score, I'm already way ahead of what they list as the 'previous state of the art'.

SEEKING INVESTMENT | Greater Seattle Area | ML, MUSIC

Project or startup: Music streaming service with AI-generated music and lyrics

Seeking for: $200,000 investment for up to 20% stake

Spending on:

- $180,000 for 1y salary for R&D

- $20,000 for hardware purchase/rent for model training

Team: just myself, software engineer with ML background

Business model: streaming subscription

Some preliminary work: Google showed neural networks can be used to generate music in MIDI format ( https://magenta.tensorflow.org/ ),
own project shows GPT-2 can generate song lyrics ( http://billion.dev.losttech.software:2095/ , based on the initial small 117M GPT-2 release ).
MIDI format supports lyrics tracks, so the two can be combined, producing karaoke-like experience.
There are also singing voice synthesizers on the market, which could be used to automatically generate song recordings.
Alternatively, real performers could record tracks.

For context, here's some recent computer music, both MIDI and direct audio generation:

https://magenta.tensorflow.org/perceiver-ar

Here is an older project that generated audio from lyrics:

https://openai.com/blog/jukebox/

Music has two key difficulties: the long, linear context makes it hard for most models to generate coherent works and reason about long-term higher level structure, and the global structure being learned is much more function of human preference, rather than of physical constraints.

That said, I don't think the gap is all that sizeable. Jukebox is easily comparable in quality to the image synthesis from arbitrary text of its time. Perceiver AR is not as impressive as DALL-E 2, sure, but DALL-E 2 is new, and it's plenty of evidence that music synthesis will continue to work.