Charles Bloom, a compression expert at RAD Game Tools and author of Oodle, writes about data compression and high-performance programming.
Commenters call it an excellent, in-depth resource on compression from basics to state-of-the-art, and note that it has been dormant for some time.
For those interested in the topic, I highly recommend the series of posts from Charles Bloom:
http://cbloomrants.blogspot.com/2023/07/notes-on-float-and-multi-byte-delta.html
> TLDR:
> For S16 deltas, bias by 0x8080 , for S32 deltas, bias by 0x80808080.
> De-interleaving multi-byte integers into homogenous streams can help, particularly with weaker back-end compressors.
> TLDR:
> For floats, just reinterpret F32 as U32.
> If the floats have a mix of positive and negative, convert the sign bit to two's-complement signed S32.
> Consider lossy elimination of negative zero -0.f
> If you didn't actually want the huge precision for floats near zero, a simple lossy encoding is just to do float += constant, which works for non-negative floats where you don't know the high end of the range so you can't just use fixed point.
> TLDR:
> The best way to compress numeric data that is larger than bytes (F32,F16,S32,S16) is usually to delta them in their original size integer, then de-interleave after the delta.
> Sometimes no filter or no deinterleave is best, particularly with stronger compressors, so being able to select filter on/off per-file can give big wins.
It's a terrible metric that fails basic dimensional analysis. Anytime you see a ratio of logs, you should already be suspicious. Say algorithm A is twice as fast as algorithm B. Then we'll have alpha r/R log(T)/log(t). The alpha r/R is constant, so the only interesting part is log(T)/log(t) = log(2t)/log(t) = log2/log(t) + 1. We can make this apparently unitless number take any value we want just by changing the unit used to measure time (or if a unit were fixed, by changing the amount of data used to test).
It's sort of vaguely superficially sensible, but the idea of being able to reduce comparison of compression algorithms (which fundamentally trade off speed for compression ratio) to a 1-dimensional value is laughable. Charles Bloom's Pareto frontier charts (http://cbloomrants.blogspot.com/2015/03/03-02-15-oodle-lz-pareto-frontier.html) are one of the more reasonable options.
If you are interested in compression at all, be sure to take a trip through Charles Bloom's blog [1]. It's an incredible read, he covers everything from the basics all the way through state of the art algorithms.
A great example is this post [2], where he talks about how to correctly implement a Huffman encoder/decoder. It's a lot tricker than it is made to sound in most books. For example, most Huffman codes that are used in practice are length limited, to allow the decoder to use smaller lookup tables. There are a bunch of surprisingly interesting tricks to get that to work well from the encoding side (which symbols do you choose to be smaller than they would be otherwise?).
[1] http://cbloomrants.blogspot.com/
[2] http://cbloomrants.blogspot.com/2010/08/08-12-10-lost-huffman-paper.html
Of course there is comparison with LZ4: http://www.radgametools.com/oodlemermaidselkie.htm
LZ4HC is LZ4 compressed with the slow compressor to achieve better compression. It is decompressed using regular LZ4 decompressor.
They also compare with zstd and lzma, which are the open source competition.
I don't know what "specs" you think are missing. They do say which architectures are supported, what the memory use is, whether multi-threading is supported, etc.
No public price, true.
You can read the blogs of the people who work on this code to get a feel for the caliber:
https://cbloomrants.blogspot.com/
https://fgiesen.wordpress.com/
These are known people with known reputation. The product itself is well known in the industry.
Interesting that, despite flaunting her edginess, the author never once does actually type out an unadulterated or censored "fuck".
The complaint I have with this sort of copy is not being offended, or of being obscene, or what have you--it's that the sort of diction used tends to make me think that whoever wrote it is simply not that bright.
If you'd like a good example of decently-written swearing, look at Zed Shaw ( http://programming-motherfucker.com/ ), jwz, or Charles Bloom ( http://cbloomrants.blogspot.com/ ).
Those folks clearly rage early and rage often, but they have some skill and class.
Isn't the magic kernel simply a 2× bilinear downscale filter? For example, if you use 2× bilinear downscaling in Tensorflow (as opposed to bilinear downsampling which would end up being just a box filter) using tf.image.resize it uses the magic kernel [1,3,3,1].
There is an article [0] that points this out, although later it goes into recommending odd filters, which insidiously break many applications where you would want to use downscaling by shifting the image by a fraction of a pixel.
[0] https://cbloomrants.blogspot.com/2011/03/03-24-11-image-filters-and-gradients.html