Comment by quotemstr
8 years ago
I have to note that none of the projects you suggested, all of which are good and useful, will do anything to address cpython startup latency problem under discussion. Why shouldn't cypthon be better?
There's also no reason to believe that startup improvements would make the interpreter incomprehensible; the unstated assumption that improvements in this area must hurt hackability is interesting. IME, optimizations frequently boost both simplicity and performance, usually by unifying disparate code paths and making logic orthogonal.
I think you misunderstood the point. These weren't things that would address the cpython startup problem - these were other priorities that can be worked on, instead of (or in addition to) the latency problems under discussion.
Saying yes to fixing one thing usually means saying no to all the other things you can be doing with your time instead. Unless you're lucky and can "kill 2 birds with 1 stone".
> - Write compilers from well-typed Python to native code.
That is one thing I really want to happen, because I think it begins to open up python to the embedded space. Micropython is nice, but it still needs an interpreter embedded into it.
There have been plenty of attempts to compile Python to faster code (usually by translating to C).
Cython can use type annotations and type inference to unbox numbers for faster numerical code, but uses ordinary Python objects otherwise. http://cython.org
ShedSkin translates a restricted (statically typeable) subset of Python to C++. https://shedskin.github.io
RPython has a multi-stage approach where you can use full dynamic Python for setup, but starting from the specified entry point, it is statically typed. Since it was created for PyPy, it comes with support for writing JIT compilers. https://rpython.readthedocs.io
Pythran translates a subset of Python with additional type annotations to C++. http://pythran.readthedocs.io
In general, the major hurdle for all attempts to compile Python to native code is that Python code is dynamically typed by default.
3 replies →
> Why shouldn't cypthon be better?
The point is that "better" is almost never a well-defined direction unless you only consider a single use-case. It's almost always a tradeoff, especially in a widely used project.
A language is a point on a landscape of possible language variants, and "better" is a different direction on that landscape for every user.
> cpython startup latency problem
The problem under discussion is that projects that currently use CPython have slow startup. One potential solution is for those projects not to use CPython. (Certainly it's not the only potential solution, but, a language that tries to be all things to all people isn't going to succeed. Python has so far done an extraordinarily good job of being most things to all people, with "I want native-code performance" being one of the few out-of-scope things.)
>IME, optimizations frequently boost both simplicity and performance, usually by unifying disparate code paths and making logic orthogonal.
I would really like to see some examples where this is the case. Optimizations in my experience have made systems more brittle, less portable and ultimately less maintainable.
Programs a fast when they don't force the computer to do much stuff, so speed overlaps (albeit imperfectly) with simplicity of code. Mostly this explains why programs start fast and then slow down as they cover more use-cases. But some optimisations also amount simplifying an existing system.
For example: as you get to know your use-cases better you might simplify your code to sacrifice unwanted flexibility. Or you might replace a general-purpose data structure with a special purpose one that not just faster, but concretely embodies the semantics your desire.
A case, that is not quite a simplification is removing code re-use. Instead of using function in three different ways, you use three separate optimised functions. Now changes to one use case don't cause bugs in the others. That's the kind of thing that quotemstr meant by "making logic orthognal".
>A case, that is not quite a simplification is removing code re-use. Instead of using function in three different ways, you use three separate optimised functions. Now changes to one use case don't cause bugs in the others. That's the kind of thing that quotemstr meant by "making logic orthognal".
And which is what I mean by making systems more brittle, less portable and less maintainable.
You find a corner case in the original function that's not covered, now instead of fixing it in one place you need to fix it in three places with all the headaches that causes.
So the next maintainer thinks: "Gee I can fix this by bringing all these functions together".
So if they're using an oo language they make an abstract base class from which the behaviour is inherited, or a function factory otherwise.
So now you're back to a slow function with even more overhead, that's even harder to debug.
So the next maintainer comes around and thinks: "Gee I can speed this up if I break out the two functions that are causing 90% of the bottleneck".
Now you have 4 completely independent functions to keep track of.
Repeat ad-nauseum.