Comment by BiteCode_dev
16 hours ago
Other design decisions that made uv fast:
- uncompressing packages while they are still being downloaded, in memory, so that you only have to write to disk once
- design of its own locking format for speed
But yes, rust is actually making it faster because:
- real threads, no need for multi-processing
- no python VM startup overhead
- the dep resolution algo is exactly the type of workload that is faster in a compiled language
Source, this interview with Charlie Marsh: https://www.bitecode.dev/p/charlie-marsh-on-astral-uv-and-th...
The guy has a lot of interesting things to say.
> uncompressing packages while they are still being downloaded
... but the archive directory is at the end of the file?
> no python VM startup overhead
This is about 20 milliseconds on my 11-year-old hardware.
Using the -S (“isolated”) flag can maybe cut that in half.
HTTP range strikes again.
As for 20 ms, if you deal with 20 dependencies in parallel, that's 400ms just to start working.
Shaving half a second on many things make things fast.
Althought as we saw with zeeek in the other comment, you likely don't need multiprocessing since the network stack and unzip in the stdlib release the gil.
Threads are cheaper.
Maybe if you'd bundle pubgrub as a compiled extension, you coukd get pretty close to uv's perf.
> real threads, no need for multi-processing
parallel downloads don't need multi-processing since this is an IO bound usecase. asyncio or GIL-threads (which unblock on IO) would be perfectly fine. native threads will eventually be the default also.
Indeed, but unzipping while downloading do. Analysing multiple metadata files and exporting lock data as well.
Now I believe unzip releases the GIL already so we could already benefit from that and the rest likely don't dominate perfs.
But still, rust software is faster on average than python software.
After all, all those things are possible in python, and yet we haven't seen them all in one package manager before uv.
Maybe the strongest advantage of rust, on top of very clean and fast default behaviors, is that it attracts people that care about speed, safety and correctness. And those devs are more likely to spend time implementing fast software.
Thought the main benefit of uv is not that it's fast. It's very nice, and opens more use cases, but it's not the killer feature.
The killer feature is, being a stand alone executable, it bypasses all python bootstrapping problems.
Again, that could technically be achieved in python, but friction is a strong force.