Jane Street
3 HN recommenders · recommended in 4 months, 2024–2025
40:34 · 108K views · Oct 23, 2024
Charlie Marsh of Astral explains how uv, a Rust Python package manager, got so fast. He covers allocators, concurrency, zero-copy tricks and the challenges of Python packaging.
Recommended as a fascinating technical account of the optimizations behind uv's speed.
4 comments from Hacker News
I recently watched a talk by the author of uv that was surprisingly fascinating [1]. He goes into a few of the more notable hacks that they had to come up with to make it as fast as it is. The most interesting thing for me was that package resolution in python given constraints defined (eg. in requirements.txt) maps to a boolean satisfiability problem which is NP-complete. So uv uses a custom SAT solver to do this. I totally under-appreciated how much goes into this software and I'm bummed I have to use Poetry at work after having watched this talk. [1] this video edit: NP-complete not NP-hard
To be fair, the whole post isn't very good IMO, regardless of ChatGPT involvement, and it's weird how some people seem to treat it like some kind of revelation. I mean, of course it wasn't specifically Rust that made it fast, it's really a banal statement: you need only very moderate serious programming experience to know, that rewriting legacy system from scratch can make it faster even if you rewrite it in a "slower" language. There have been C++ systems that became faster when rewritten in Python, for god's sake. That's what makes system a "legacy" system: it does a ton of things and nobody really knows what it does anymore. But when listing things that made uv faster it really mentions some silly things, among others. Like, it doesn't parse pip.conf. Right, sure, the secret of uv's speed lies in not-parsing other package manager's config files. Great. So all in all, yes, no doubt that hundreds of little things contributed into making uv faster, but listing a few dozens of them (surely a non-exhaustive lists) doesn't really enable you to make any conclusions about the relative importance of different improvements whatsoever. I suppose the mentioned talk[0] (even though it's more than a year old now) would serve as a better technical report. [0] this video
There is a talk about it from one of the authors here: this video tl;dw Rust, a fast SAT solver, micro-optimisation of key components, caching, and hardlinks/CoW.
So much love for @burntsushi He's improved my life in so many ways, rg, ... and now also in Python with uv! Glad to see Charlie giving him the credit for figuring out the version comparison optimization in uv in [1]. Also huge thank you to the whole Astral team for all they're doing on uv It's magic! 1: this video