Google AI Blog

I think it's processing power and engineering effort. I got a Sony RX100M2 for my mom and it has the same computational photography techniques that Google released the following year (https://ai.googleblog.com/2014/10/hdr-low-light-and-high-dynamic-range.html). But Sony's image stacking is only in "Superior Auto" mode, and is only used when necessary. Google's implementation does a lot of advanced work, including selecting and blending parts of the photo depending on motion, that Sony doesn't do. I assume Sony's imaging engineers have less expertise in advanced processing, and didn't have the resources to implement the features that Google did. Sony also has to devote engineering resources to other features - a lot of their sales are to photographers that will edit in post (RAW), and later, videographers. So features that are only in "auto" mode may have limited budget.

There's ample literature showing that neutral vocoders generalize to new speakers not seen in training. IME, once you train on around ten speakers the models generalize quite well to new speakers, and in practice training sets for production models are often using hundreds of speakers for training.

There are also non autoregressive GAN vocoders which work great, with a tiny fraction of the compute of the original WaveNet. Soundstream is a good example of this.

My own work on bird song separation uses non AR models for generating separation masks. It works great and generalizes to new habitats and species without difficulty. https://ai.googleblog.com/2022/01/separating-birdsong-in-wild-for.html?m=1

Calling architecture variants a hack is risking no true Scotsman arguments... The only true neutral networks are the ones that don't work, perhaps?

I highly recommend SF. The trick is you must use the books via coqIDE or something equivalent; viewing the rendered HTML is almost pointless at best, but probably counter-productive.

Binary search would be a good exercise. I think SF volume 1 would be more than enough, with the exception of deciding how to model arrays (you'd probably want to look up Coq's persistent arrays/PArray system for this.) SF volume 3 does binary search _trees_ which is similar but avoids the array problem (as is customary in functional languages.)

However, Coq (by default) also uses infinite precision numbers. This works very well for proofs but isn't necessarily realistic. This is where some extra fun comes in: fixed precision arithmetic is a very common source of bugs for binary search: https://ai.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html

After reading vidarh's comment below ("The biggest challenge with evaluating all of these, is that once you've listened to a comprehensible version of one of these samples, they all sound more intelligible."), I recommend trying to listen to the Lyra version first, then listen to the original next. It actually does sound very strange when you don't have the true audio parsed in your head already. I'm not sure if I would like having to do the extra work to translate this in my head on every conversation.

It's more apparent in the video example from the Google post, I won't spoil it but there's a word that starts with B that sounds very funny in the Lyra version (listen to that one first): https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-codec-for.html

>However it is still just pattern matching

the visual cortex in your brain is also "just a pattern matching" system. guess it's not very impressive by your standard.

This[1] isn't my example (it's from another HN user), but if you work as a programmer and you're not absolutely jaw on the floor astonished by this example then I don't know what to say.

Explaining[2] the emergent behaviour is literally cutting edge research. Hand waving this behaviour away as just "probabilistically generating a likely next word" is ignorant.

It's amazing in similar ways to Conway's Game of Life.

[1] https://imgur.com/HOEnxYb

[2]https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html

> The chip design itself should be the secret sauce. Not the tools you make the chip with.

The secret sauce is generally whatever gives one a competitive advantage. Businesses typically open things up when the want to reduce the cost of something and/or cause pain for someone else (i.e. killing their cash cow), not because they're benevolent and want to share.

> I challenge you to answer your own question in reverse: are any companies other than Nvidia embarking up AI/ML chipmaking in a closed fashion? There probably are, let's follow & watch them.

Isn't Google's floorplanning work closed source? https://ai.googleblog.com/2020/04/chip-design-with-deep-reinforcement.html