Comment by dima-quant
8 hours ago
True, nimic is a statically typed Python subset without much of its dynamism with the aim to be foremost an efficient systems language. Though in nimic, without using isinstance, the instance-based dispatch is realised via variant types. Yes, emulating the Nim constructions in Python was the hardest part, while making the transpiler was straightforward.
Indeed, the AOT compilation leads to great speed-ups for heavy custom numeric calculations that cannot be easily vectorized in numpy, such as the raytracing logic. In cases when most of calculations are performed in an external module (written in e.g. C, Rust or Cython etc) the performance gain might be much less, but the added value here is that the high performance module itself can be written in nimic, keeping the codebase purely in Python and consolidating the codebase.
When starting with a "pythonic" code, rewrites in nimic can be substantial but so would be a rewrite in a systems language like C or Rust. Besides allowing to optimise the fast path, nimic provides low level functionality, such as pointers to pointers and bitwise operations that actually executes within CPython, for example, as in mp4 muxer implementation in dima-quant/ndsl_raytracer/src/nraytracer/minimp4.py
the variant-types-for-dispatch thing is a nice tradeoff, makes sense. honestly the strongest pitch in there is the "write the hot module in nimic instead of dropping to c/rust/cython" angle, which is basically the cython niche. so the question is what nimic adds over cython for that. my guess is dual-mode: a .pyx isn't valid python and won't run under plain cpython, but nimic stays importable and debuggable in cpython and you compile only when you want speed. if that's the edge i'd put it front and center, it's a way sharper sell than "python but fast".
one thing i'd worry about with "runs unmodified in cpython" though: python ints are arbitrary precision and nim's aren't. so the same code can give you a bignum under cpython and a wraparound under the compiled path. how do you handle that, or is matching cpython semantics explicitly a non-goal for the typed subset?
> how do you handle that
It looks like (please correct me, OP, if I'm wrong) it works the other way around: you use sized int types in Nimic code, and their semantics are emulated on the Python side. See here: https://github.com/dima-quant/nimic/blob/main/src/nimic/ntyp...
So I'd say in Nimic Python you get Nim-style integer emulation when run from Python, keeping both paths consistent with each other - but breaking consistency with the rest of Python. Which is OK, I think, given it's explicitly a subset(s) of the language(s). It would be possible to make `int` transpile to some BigInt Nim implementation, but you'd need an external dependency for this, as they are not in Nim's stdlib. However, in the speed-focused context, I'm not sure if defaulting to BigInt every time the compiler sees an `: int` annotation would work well. It's a hard decision to make. Curious what's the OP opinion here?