Comment by ajb
9 hours ago
It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss aversion": we dislike losing something we have more the value we put on gaining it. In a new language each newly added feature is a gain, but in a language subset you're always encountering things you would have in the full language. So no matter how good the rational case is, there is a bias against them. I can't think of a successful one actually - anyone?
The author seems to speak to that here:
> SPy does something different: on one hand, it removes the dynamic features which make Python "slow", but on the other hand it introduces new features which make it possible to implement and use the same pythonic patterns which we like.
The author seems unusually focused on explaining his new way, which may help it feel more like a new language with a coherent philosophy and unique benefits in its own right.
Not unlike how Crystal isn't really "compiled Ruby" but rather its own language that just looks and feels similar to Ruby.
> in a language subset you're always encountering things you would have in the full language
Exactly. I once pitched the idea of a Python subset (with a different focus, not performance like SPy), and almost every reaction was "will it support <favourite Python feature / favourite library>".
For example, a new language can build its own solution for array math, or maybe that's not something its users need. OTOH many consider a Python subset to be unacceptable if it doesn't specifically support NumPy.
In the end I came to agree with https://pointersgonewild.com/2024/04/20/the-alternative-impl...:
> positioning your project as an alternative implementation of something is a losing proposition
> don't go trying to create a subset of Python [...] Do your own thing. That way, you can evolve your system at your own pace and in your own direction, without being chained by expectations that your language should have to match the performance, feature set, or library ecosystem of another implementation.
I can see the wisdom in this.
As a counter-example, it feels as though Typescript has managed to (largely?) succeed as a subset of Javascript, so maybe the "do your own thing" isn't entirely a lost-cause -- maybe it's just really really difficult?
Typescript is a superset of javascript
1 reply →
One of the standard architectures for complex applications is to put together a scripting language and an systems programming language. You can either look at the scripting language as primary with the systems language used for performance or hardware interfacing or you can look at the systems language being primary and the scripting being used around the edges.
The model of having a closely related scripting and systems language would be an optimization of this and SPy seems like an answer.
[1] AAA games that have a scripting engine on top of C/C++, a finite element solver that lets you set up a problem with Lua and solve it with FORTRAN, etc.
Yes, there are quite many real-world cases of this architecure. But, wouldn't it be better it the same language (more or less) can be used for both? I don't think such a language exists currently, but I think it would be a nice goal.
Oxcaml, of course.
1 reply →
I would say that Rpython is successful. However, it's primary goal is to be whatever the PyPy developers need to write PyPy.
I totally agree about subsets though, I would much rather have a superset like Cython, but that has it's own challenges for compatibility if you wanted to have a "pure python" version of the same library. Which is why I really like that Cython is using typehints now.
Yeah Rpython escapes this as most people don't write it directly.
Micropython is another example. No-one expects micropython to support numpy. They're just happy to get away from C. But where you can use the full python, you wouldn't use micropython.
Throwing my hands up and moving to Nim was downright easy next to the excessive effort I put into trying out Nuitka, Numba, and PyInstaller for my use case. If you want static compilation, use a language and libraries built with that assumption as a ground rule. The herculean effort of building a half-compatible compiler for a dynamic language seems like a fool's errand, and would be a fun curiosity if so many people hadn't already tried it, especially with Python.
I was looking for someone else that had done this, I had the same exact experience.
That said, anyone looking into a completely static typed language that has nice ergonomics, is easy to pick up but has enough depth to keep you busy for weeks on end, and is versatile enough to be used for anything, do yourself a favor and give Nim a try.
https://nim-lang.org/
It's so common for something like this to struggle for decades before succeeding.
For instance people have been trying to make a memory safe C for a long time, just recently we got
https://fil-c.org/
which has high compatibility and relatively good performance for that kind of thing. The strength of Python is that so many people are trying things with it.
all of this is well and good if you completely forget that there are billions of lines of Python in prod right now. so your grand epiphany is basically on the level of "let's rewrite it in Rust". i'll let you hold your breath until that rewrite is done (and in the meantime i'll explore workarounds).
Now there are billions of lines of python in production. But it wasn't so long ago that it seemed like the entire industry was going to completely standardise on C++, and if you wanted a picture of your future, it would be grinding away debugging buffer overflows and thread lockups - forever.
I kind of object to this take.
Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience.
This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy.
OTOH I do understand the weight of a currently existing corpus in production, evidence is the ton of COBOL code still running. But still, your reply kind of sucks.
I finished the rewrite in just a few months and have been happily maintaining it for two years and extending it with powerful features it never had before, partially thanks to optimized native binary speed, partially thanks to the ludicrous compatibility of a musl-libc static binary. The rewrite is the backend foundation of a web product stack that is best-in-category.
I didn't choose Nim because I was an evangelist; I had only toyed with it twice before. After much evaluating (and wrestling an ultra-frustrating previous attempt at a Rust rewrite my predecessor had tried), Nim surfaced as the best tool for the specific job. And it still is. It doesn't have to be our entire product stack; it's doing a small subset of very important jobs, doing them well, and is invoked by other languages that do their jobs better as web servers and such. A modular stack helps each language and tool shine where it works best.