Jean-Marc Valin, an audio and video codec developer behind Opus and Daala, publishes demos and write-ups on codecs and neural audio.
HN links his demos for RNNoise noise suppression, LPCNet speech coding and Daala's perceptual vector quantization as examples of combining neural nets with signal processing.
> Therefore, on average, unless the lossless algorithm has a perverse encoding that uses more bits than necessary to encode the possible outputs, no lossless algorithm can beat it on average
I think you mean "unless the lossy algorithm has a perverse encoding."
But your argument proves too much. It's true that no lossless algorithm can beat a lossy algorithm if averaged across all possible images, for precisely the reason you say. But for the same reason, no lossless algorithm can beat the "compression algorithm" of no compression at all, like BMP but without redundancy in the header, averaged across all possible images.
The unknown question — which I think we don't even have a good estimate of — is how much incompressible information is in the images we're interested in compressing. Some of that information is random noise, which lossy algorithms like JPEG can win by discarding. (Better algorithms than JPEG can even improve image quality that way.) But other parts of that information are the actual signal we want to preserve; as demonstrated by Deep Dream, JPEG doesn't have a particularly good representation of that information, but we have reason to believe that on average it is a very small amount of information, much smaller than JPEG files.
If it turns out that the actually random noise is smaller than the delta between the information-theoretic Kolmogorov complexity of the signal in the images we're interested in compressing and the size of typical JPEG files…
One thing I'd love to see follow from this is an image format based on its intra coding. The companies involved expressed interest (https://www.cnet.com/news/google-mozilla-av1-photo-format-could-outdo-aging-jpeg/), and tests (https://people.xiph.org/~tdaede/av1stilldemo/) look great.
The .heic format now used by iPhones is the same idea (https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format). But wider use of that is going to be constrained by the cost problem HEVC has in general (two patent pools to deal with). Some of WebP is based on VP8 (not VP9) intra coding too.
There are a lot of tools packed in AV1's intra coding (and HEVC's, though I've read less about that). Block sizes range from 4x4 to 64x64 and there's a mix within one image, so the encoder can use the right size for the level of detail in each area. There are more ways to predict a block's content from what's to the top and left, which leaves less work for the JPEGish DCT part. There's clever de-ringing post-processing that, in effect, blurs away many of the attention-getting JPEG-y DCT artifacts around edges, while 1) being aware of the direction of the main edge itself to avoid blurring that away and 2) using contrast thresholds to preserve as much other legitimate detail as it can--more about deringing at https://people.xiph.org/~jm/daala/deringing_demo/ .
(There's some good detailed discussion at https://parisvideotech.com/wp-content/uploads/2017/07/AOM-AV1-Video-Tech-meet-up.pdf and in…
There's a few recent papers actually that show minor improvements by integrating LPC prediction into deep methods ([0], [1]). In my experience (some of which comes from reproducing these, some of which comes from my own experiments), this isn't actually too useful, at at most offers a minor modeling benefit.
The main difference between something like Festival and what we have now is the amount of domain-specific engineering. (This is generally the promise of deep learning -- replace hand-engineered features with simple-to-understand features and a deep model.) If you go and read the Festival manual, you're going to find tons of domain-specific rules and heuristics and subroutines; for example, there's a page on writing letter to sound rules as a grammar [2]. Nowadays, we may have a pipeline that resembles Festival at the high level, but each step of the pipeline is learned as a deep model from data rather than being carefully hand-engineered by many people over the course of years. This yields much more fluid speech as well as much, much faster iteration and experimentation times, leading to faster progress as well.
[0] https://arxiv.org/abs/1811.11913
[1] https://people.xiph.org/~jm/demo/lpcnet/
[2] http://www.festvox.org/docs/manual-2.4.0/festival_13.html#Letter-to-sound-rules
> SILK that seemed to be the parent of OPUS
That's right - SILK was a codec tuned for voice chat, not music. CELT is the other parent of OPUS and is tuned for music. (Opus uses a hybrid strategy that switches between codecs as necessary.) Don't just look at the benchmarks, at the bottom of the page you can click on the links and listen to a bunch of 64 kbps Opus samples. They're very, very good.
And no, they aren't indistinguishable from a CD. As I said in the original comment, they're more than good enough for high quality music streaming for a video chat.
> I'd really love to find out that codecs have halved audio size in the past half decade.
This has happened, to my ears at least. A little over a decade ago, I could not-infrequently tell a V2 MP3 (~196 kbps) from the FLAC. Took 256 kbps (~V0) to be reliably transparent. (I think it's improved marginally since then, V2 might be transparent almost all the time now.) These days, it's quite infrequent I can tell a 128 kbps Opus file from the FLAC. And the best codecs are quite passable even at 64 kbps.
Actually, Opus is "okay" even down to like 48 kbps. Try the sample here: https://people.xiph.org/~jm/opus/opus-1.2/
There was a blog post a few months ago: [0]
It looks like there have been a few dead ends -- clever ideas that for one reason or another didn't pan out -- but overall, it's making solid progress. It remains, however, very experimental and is far from complete.
One area in which Daala works very well right now is in compression of still images. There's a demo at the bottom of the page which shows it as much better than HEVC for that purpose, and you can also see how the codec has improved over time. Of course, the relative goodness of codecs isn't necessarily consistent over widely varying bitrates.
In some of the earlier posts the Daala researchers expressed the belief that the quantitative metrics often don't accurately correspond to subjective perception of quality. Netflix seems to be putting a lot of work into developing better metrics [1], but the Daala researchers seem to rely on subjective A-B tests as the 'gold standard'.
[0]: https://people.xiph.org/~jm/daala/revisiting/
[1]: http://techblog.netflix.com/2016/06/toward-practical-perceptual-video.html
> Interesting to see they're also applying Daala to still images. It's a very different set of constraints and optimizations than video.
Video codecs have to encode keyframes, which is roughly the same problem as still images. Many still image codecs are modified intraprediction modes from video codecs. So it's really a subset[0] of the constraints and optimizations, not a totally different set.
It's also the case we've recently spent a lot of time working on intraprediction issues (and related topics). One reason for that is that the lapping filters make intraprediction very difficult compared to traditional codec designs.
Demo2[1] explains much of this difficulty in the context of a solution that we tried and had to abandon. The paint demo[2] explains another solution we tried, which also failed, but which is now the basis of a post-filtering technique.
[0] Modulo things like color format support, metadata, etc.
[1] http://people.xiph.org/~xiphmont/demo/daala/demo2.shtml
[2] https://people.xiph.org/~jm/daala/paint_demo/