Comment by ajross
15 days ago
> PyPy is a fantastic achievement and deserves far more support than it gets
PyPy is a toy for getting great numbers in benchmarks and demos, is incompatible in a zillion critical ways, and is basically useless for large-scale development for anything that has to interoperate with "real" Python.
Literally everyone who's ever tried it has the experience that you mock up a trial for your performance code, drop your jaw in amazement, and then run your whole app and it fails. Until there's a serious attempt at real 100% compatibility, none of this is going to change.
Also none of the deltas are well-documented. My personal journey with PyPy hit a wall when I realized that it's GC is lazy instead of greedy. So a loop that relies on the interpreter to free stuff up (e.g. file descriptors needing to be closed) rapidly runs into resource exhaustion in PyPy. This is huge, easy to trip over, extremely hard to audit, and... it's like it's hidden lore or something. No one tells you this, when it needs to be at the top of their front page before your start the port.
"Ask HN: Is anyone using PyPy for real work?" from 2023 contradicts you about PyPy being a toy. The replies are noticeably biased towards batch jobs (data analysis, ETL, CI), where GC and any other issues affecting long-running processes are less likely to bite, but a few replies talk about sped-up servers as well.
https://news.ycombinator.com/item?id=36940871 (573 points, 181 comments)
Timely management of external resources is what the `with` statement has been for since 2006, added in python 2.5 or so. To debug these problems Python has Resource Warnings.
Additionally, CPython's gc is also only eager in a best effort kind of way. If cycles are involved it can take long to release memory. This will become even more the case in future versions of CPython, in the free threading variants.
Sorry, the with statement is non-responsive. The question isn't whether you "can" write PyPy-friendly code. Obviously you can.
The question isn't even whether or not you "should" write PyPy-friendly code, it's whether YOU DID, or your predecessors did. And the answer is "No, they didn't". I mean, duh, as it were.
PyPy isn't compatible. In this way and a thousand tiny others. It's not really "Python" in a measurable and important way. And projects that are making new decisions for what to pick as an implementation language for the evolution of their Python code have, let's be blunt, much better options than PyPy anyway.
Strongly disagree. If you're relying on Python garbage collection to free file descriptors in a loop, you have a subtle bug that will rear its head in unexpected and painful ways (and by some unwritten law of software, most notably either at 3 AM or when you have an important demo scheduled). This is true whether you're running in CPython or PyPy. It's not hard to avoid - use `with` or `try...finally`. It's not some newfangled language feature. It's not a surprise - you can't write good RAAI code in Python. It's a sign of someone with a poor grasp of the language they're using. If you find things like this, you should fix them, even if you never intend to use PyPy.
2 replies →
I've run into similar resource limit exhaustion due to the GC not keeping issues with cpython as well