Ash Vardanian, creator of USearch, writes about low-level performance work: SIMD, GPUs, string processing, vector search, and C++ and Rust parallelism.
HN links it for performance deep dives on Graviton SVE, trigonometry speedups, and USearch molecule embeddings; one commenter reported a broken table-of-contents link.
Three fundamental flaws of SIMD ISA:s - Bits'n'Bites (https://news.ycombinator.com/item?id=43783416)
1. Snell-Pym » Processor architecture (https://www.snell-pym.org.uk/archives/2017/03/17/processor-architecture/)
2. Fun with vectors in the Raspberry Pi 1 - Part 9 (https://thinkingeek.com/2021/08/22/raspberry-vectors-part-9/)
I think we have enough pieces of machinery working already that we can start with the most exciting part of this journey: autovectorisation!
3. Efficiently compiling efficient query plans for modern hardware | the morning paper (https://blog.acolyer.org/2016/05/23/efficiently-compiling-efficient-query-plans-for-modern-hardware/)
4. Understanding SIMD: Infinite Complexity of Trivial Problems ?? | Ash's Blog (https://ashvardanian.com/posts/understanding-simd-complexity/)
5. SIMD in the GPU world - RasterGrid (https://www.rastergrid.com/blog/gpu-tech/2022/02/simd-in-the-gpu-world/)
It’s a nice post, but “using array methods” probably shouldn’t be placed in the “Efficient Implementation” section. As often happens with high-level languages, a single plain old loop is faster than three array methods.
Similarly, if you plan to query those vectors in search, you should consider continuous `TypedArray` types and smaller scalars than the double precision `number`.
I know very little about JS, but some of the amazing HackerNews community members have previously helped port SimSIMD to JavaScript (https://github.com/ashvardanian/SimSIMD), and I wrote a blog post covering some of those JS/TS-specifics, NumJS, and MathJS in 2023 (https://ashvardanian.com/posts/javascript-ai-vector-search/).
Hopefully, it should help unlock another 10-100x in performance.
ARMs Neoverse IP does support SVE, so it's at least already relevant in cloud applications. Apparently AWS Graviton3 has 256bit vectors, but Graviton4 regressed back to 128bit vectors for some reason?
https://ashvardanian.com/posts/aws-graviton-checksums-on-neon-vs-sve/
To accelerate the analytical solution, there are more obvious techniques, like replacing trigonometric opetations with series. This is a 10x improvement in https://ashvardanian.com/posts/google-benchmark/#slow-trigonometry-in-libc-and-stl-and-fast-math
Nice project! A regular on HN and creator of usearch built an embedding search for the same dataset and did a write up which is a great read.
https://ashvardanian.com/posts/usearch-molecules/
If the author of the article sees this, this link in the table of contents doesn't work:
https://ashvardanian.com/posts/python-c-assembly-comparison/#vnni