Comment by zahlman
16 hours ago
> No, every code path you don't execute is that.
Even in compiled languages, binaries have to get loaded into memory. For Python it's much worse. On my machine:
$ time python -c 'pass'
real 0m0.019s
user 0m0.013s
sys 0m0.006s
$ time pip --version > /dev/null
real 0m0.202s
user 0m0.182s
sys 0m0.021s
Almost all of that extra time is either the module import process or garbage collection at the end. Even with cached bytecode, the former requires finding and reading from literally hundreds of files, deserializing via `marshal.loads` and then running top-level code, which includes creating objects to represent the functions and classes.
It used to be even worse than this; in recent versions, imports related to Requests are deferred to the first time that an HTTPS request is needed.
> binaries have to get loaded into memory.
Unless memory mapped by the OS with no impact on runtime for unused parts?
> imports related to Requests are deferred
Exactly, so again have no impact?
rtld does a lot of work even in “static” binaries to rewrite relocations even in “unused parts” of any PIE (which should be all of them today) and most binaries need full dyld anyway.
> Unless memory mapped by the OS with no impact on runtime for unused parts?
Yeah, this is presumably why a no-op `uv` invocation on my system takes ~50 ms the first time and ~10 ms each other time.
> Exactly, so again have no impact?
Only if your invocation of pip manages to avoid an Internet request. Note: pip will make an Internet request if you try to install a package by symbolic name even if it already has the version it wants in cache, because its cache is an HTTP cache rather than a proper download cache.
But even then, there will be hundreds of imports mainly related to Rich and its dependencies.
> Only if your invocation of pip manages to avoid an Internet request.
Yes it does, by definition, the topic of discussion is the impact of unused code paths? How is http cache relevant here? That's a used path!
1 reply →