Casey Primozic writes about neural networks, web visualizations, Rust and WebAssembly, and procedural game development.
HN commenters link it as a personal portfolio site and cite his posts on a signal analyzer, Geotoy, and neural network experiments.
This year, some members of the Rust Programming Language Community Server on Discord set out to solve AoC in under 1ms. I'm pleased to announce that through the use of LUTs, SIMD, more-than-questionable `unsafe`, assertions, LLVM intrinsics, and even some inline ASM that goal has been reached (for real this time)!
*As of today, our solutions are able to solve all 49 problems in <1ms!*
I have obtained consent from all the top participants to post their solutions to a shared repo, for the community to review and learn from! *All solutions are now available at the linked GitHub repo!*
Our solutions have a total runtime of *988936ns*!
# Context/Caveats
- All submissions were run on the same hardware (Ryzen 5950X) to ensure consistency, with the same compiler flags and features available. This was on rustc nightly (updated throughout the course of the contest), and with CPU speed capped at 3400 MHz with boost clock disabled.
- AVX-512 was not available on the machine so none (?) of the solutions utilize that particular set of accelerated instructions, but there is plenty of other SIMD in use.
- All submissions were run against the same inputs to ensure consistency.
- Caching anything that has been fed with input was not allowed to prevent cheating and/or trivial solutions like `Map<Input, Output>`.
- For the same reason, inputs were not directly available to the participants, and were not provided at compile-time.
- Participants were allowed to use compile-time tricks…
>People keep calling it "next next token predictors", but clearly there is something more going on and I would love for someone to give a simple explanation.
Next token prediction is the objective function. The model is asked to predict the next word yes but it's also allowed to compute the answer and more importantly, the entire training process is supposed to be the model learning and figuring out what sort of computations aid the prediction of the corpus it's trained on.
If your corpus is language A followed by the translation in Language B then there's little choice but for the model to learn computations that translate as loss goes down.
Is your corpus is chess moves then again, it's going to have to learn how to compute chess games to reduce loss.
You can see this with toy models trained on toy problems. Example - a tiny transformer trained on addition examples - x + y = z learning an algorithm for addition.
https://cprimozic.net/blog/reverse-engineering-a-small-neural-network/
"Pick the right word" is not a trivial exercise for the vast majority of text data.
And again because people often make this mistake but a LLMs ultimate objective is NOT to produce "text that looks right" but "text that is right". Of course "right" as determined by the training corpus but basically anytime it picks a wrong word is opportunity for the model to learn and learn it does.
I like the meme of the personal webpage, which has sort of fallen by the wayside with the rise of social media.
It's one of those ideas that spreads because you see other people doing it: You get the OG blogs still up with 90s styling, student blogs with like 2 posts, and longform explorations of trying to squeeze out every ms of performance from D3 by rewriting parts of it in WASM [1], which link to the posts that it was inspired by, an even crazier attempt which tries to turn off all its safeties.
It forms an entire ecosystem, to which Hackernews, Google, et etc, form a kind of gateway into the canopy of a blogosphere.
One more thing: can we do more linking? Like, have a section of our blogs that's basically, "here are the blogs of a bunch of people I find interesting"? Is there a reason why we don't do that (spam or harassment)?
1. https://cprimozic.net/blog/speeding-up-webcola-with-webassembly/
> LLMs by design are not capable of reason.
This isn't true.
A deep neural network certainly can emulate the logical functions we think of as "reasoning" (ie, AND/OR/XOR functions).
See for example:
https://cprimozic.net/blog/boolean-logic-with-neural-networks/
https://www.cs.toronto.edu/~axgao/cs486686_f21/lecture_notes/Lecture_08_on_Neural_Networks_1.pdf
https://towardsdatascience.com/emulating-logical-gates-with-a-neural-network-75c229ec4cc9
https://medium.com/@stanleydukor/neural-representation-of-and-or-not-xor-and-xnor-logic-gates-perceptron-algorithm-b0275375fea1
How does Tinygrad fall short? Performance is fine [0]. It's much smaller than Pytorch and all, but that's kind of in the name.
I've been hearing about MLIR and OpenXLA for years through Tensorflow, but I've never seen an actual application using them. What out there makes use of them? I'd originally hoped it'd allow Tensorflow to support alternate backends, but that doesn't seem to be the case.
0: https://cprimozic.net/notes/posts/machine-learning-benchmarks-on-the-7900-xtx/
https://cprimozic.net/
Personal site and blog. Designed with the goal of being what I want people to see if they were to google me.
My favorite part is my portfolio: https://cprimozic.net/portfolio/
It contains descriptions + screenshots of my significant projects going back to when I first learned programming.