Comment by rich_sasha
20 hours ago
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.
This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
I think the idea is correct, but hard to retrofit, you sort of have to have it there from the start.
Objective-Smalltalk runs on the Objective-C runtime, so Objective-Smalltalk methods look like Objective-C methods accessed via objc_msgSend() to the entire rest of the system, including to other Objective-Smalltalk code.
That means it doesn't really matter to the rest of the system if any particular method is interpreted, AOT-compiled or JIT-compiled. They can't tell, except for execution speed.
And of course the execution model includes machine primitives, which the interpreter also deals with but doesn't make faster, but the native compilers do.
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)
> I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
Python already compiles to bytecode (the .pyc files are a cache thereof). It's just that interpreting even the simple bytecode instructions can take dozens of clock cycles each. I imagine that a lot of this is not in the interpreter loop (which has seen quite a bit of optimization, IIRC up to the level of "predictive" gotos within a big switch statement within a loop) but just unwrapping objects and moving values to and from registers, dealing with the fact that there is basically never any cache coherency for anything etc.
Basically I'm not convinced that writing out the set of machine-code instructions needed to interpret the bytecodes would be a big win, because it takes too many instructions for simple tasks.
The Julia approach is the same as BASIC (as originally designed), Lisp, Prolog, as added info.
I don't recall being able to inspect compiled machine code in basic like I can in Julia.
[EDIT - I assume you are commenting on Dartmouth BASIC.]
1 reply →
The language forum is the place to start- you should already have a thread there gathering feedback before writing a pep?
Thank-you, it's a longtime idea, but I was finally able to implement it, but now I'm trying to learn the procedure
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
By the way, the current nightly (especially if considering the optimized build) has a very substantial TTFX drop compared to the current release [1]
[1] https://perf.julialang.org/?tab=overview
It is overly exaggerated, especially when taken into account how long it takes to compile native libraries that people keep calling "Python" libraries, if there isn't a suitable wheel for the current environment.
5 replies →
There’s an #internals-and-peps channel in the Python discord where some folks hang out.
That's basically how Python already compiles source to byte-code.
Julia gets a lot more benefit out of it because it's compiling type-specialized version of the function all the way down to machine code. So for example, a function that's being called with a `Vector{Int64}` ends up dealing with an array of unboxed ints, not going through dynamic dispatch on a generic array for every operation.
2 replies →
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
Good point.
Maybe also then, importing module as Cython?
compiles on first run?
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
This is largely the idea behind py2many + fquery, but we want to get rid of C-API as well.
This is numba?
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.
Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably already have and understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.
With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).
[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.
[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.
> No dicts or objects or lists or strings, etc.
That's a really intense restriction. Sounds like the name fits; it's not general purpose, it's jit for number crunching and little else.
I'm not sure exactly what you mean by "bound by the interpreter" but there's nothing about touching those data structures that requires the interpreter to be involved in most cases. The problem of slow APIs can be solved by the JIT having intimate knowledge of python's internals, can't it?
4 replies →
Last I tried numba, admittedly a long time ago, it was a pain due to extensive system requirements or alternatively requiring use of conda, which I dislike (for many reasons, but it's besides the point - I want to use standard python tools). Did they start to bundle their dependencies or..?
1 reply →
I posted it as well before scrolling down and seeing yours. Deleted mine, as your post goes into much more detail.
I don’t know how people who’ve worked with Python and been frustrated with its performance don’t know about Numba.
if you want GPU-backed/compiled numpy code then JAX is also a good option.
Not really. Numba is great at optimizing numpy code but not necessarily other general Python code.
We just need Mojo to have 99% Python friendly syntax and we can forget about CPython.
Numba does this. Generally as fast as Cython, but much easier to write.
Numba is basically this.