Comment by zahlman

> the entire python ecosystem generally does not put much emphasis on startup time.

You'd think PyPy would be more popular, then.

> even modules in the standard library will pre-compile regular expressions at import time, even if they're never used, like the "email" module.

Hmm, that is slower than I realized (although still just a fraction of typical module import time):

  $ python -m timeit --setup 'import re' 're.compile("foo.*bar"); re.purge()'
  10000 loops, best of 5: 26.5 usec per loop

  $ python -m timeit --setup 'import sys' 'import re; del sys.modules["re"]'
  500 loops, best of 5: 428 usec per loop

I agree the email module is atrocious in general, which specifically matters because it's used by pip for parsing "compiled" metadata (PKG-INFO in sdists, when present, and METADATA in wheels). The format is intended to look like email headers and be parseable that way; but the RFC mandates all kinds of things that are irrelevant to package metadata, and despite the streaming interface it's hard to actually parse only the things you really need to know.

> Because the python ecosystem doesn't generally optimize for speed (especially startup), the slowdowns end up being contagious. If you import a library that doesn't care about startup time, why should your library care about startup time? The same could maybe be said for memory usage.

I'm trying to fight this, by raising awareness and by choosing my dependencies carefully.

> you first have to have python+dependencies installed before you can install python and your dependencies

It's unusual that you actually need to install Python again after initially having "python+dependencies installed". And pip vendors all its own dependencies except for what's in the standard library. (Which is highly relevant to Debian getting away with the repackaging that it does.)

> I think it's possible to write a pretty fast implementation using python, but you'd need to "greenfield" it by rewriting all of the dependencies yourself so you can optimize startup time and bootstrapping.

This is my current main project btw. (No, I don't really care that uv already exists. I'll have to blog about why.)

> there are _some_ improvements that have happened in the standards/PEPs that should eventually make they're way into pip

Most of them already have, along with other changes. The 2025 pip experience is, believe it or not, much better than the ~2018 pip experience, notwithstanding higher expectations for ecosystem complexity.