Comment by jakobnissen
13 hours ago
Nope. Funnily enough no one can agree on why - if you ask five people, you get six answers.
My own take is that Julia didn’t since the two language problem as much as was defeated by it.
Julia didn’t attract the high-level Python data science crowd because of Julia’s latency issue, lack of package ecosystem, and the inconveniences that a high performance compiled language incurs, such as having parametric containers.
The research software engineer crowds didn’t buy in because Julia has no interfaces or automatically checkable behavior, poor static tooling, imprecise semantics which is hard to build abstraction on, and a complex performance model that makes it hard to ensure speed, and is hard to deploy.
So, where they tried to make a language that can span the gap, they succeeded in making a language that works for neither, and which no-one wants.
I like the language. But after having used it for eight years, I find it increasingly hard to argue against the point that it’s better to choose Rust for software engineering and Python for scripting.
Edit: I should say: I used it for eight years because it IS fine for my specific niche: High performance research software engineering. Where I care neither about the convenience of Python, nor need to write truly robust and maintainable code. Where my choice of language was personal and I didn’t need to convince a team of coworkers.
I like your comment.
I made many critical comments about Julia here, on this website, but they mostly boil down to clumsiness of Julia.
Does it solve "two language problem"? Kinda, but through this it is less convenient to use than Python, and harder to reason about performance of the particular piece of code than C. Yes, there is a big chance that idiomatic, straightforward Julia code will run pretty fast, but there is also big chance that it will run unexpectedly slow, and you need to know fair bit to be able to debug this... so kinda like going from Python to C?
Is dynamism and interactivity useful? Immensely, but Julia pays for it with poor AOT support (yes, even with juliac and, still experimental, --trim option).
There is also stuff that I consider unacceptable. Debugger being a separate package you need to download, and it even cannot debug compiled code so it needs its custom interpreter? This piece of crap that is Base.@enum, so anyone that wanting proper enums need to install EnumX.jl? And why the hell StaticArrays.jl even exists as a separate package if Julia puts so much focus on numerical applications?
And then we come to tooling and IDE support. Oh, boy - Julia VSC extension is such a miserable experience, and there is not much else out there.
Julia is incredibly fun to play and tinker with solo, and some stuff from SciML is straight-up awesome, but overall it is wasted potential, killed by thousand cuts.
There is now a Julia plugin for JetBrains IDEs: https://plugins.jetbrains.com/plugin/29356-flexible-julia
> it’s better to choose Rust for software engineering and Python for scripting.
Rust and Python work in some scenarios but not all. Unless the native components are all developed already, you will need a Rust programmer on your team. Rust expertise may be available to organizations of a sufficient size and a narrow focus. In my experience, this sort of arrangement requires a team with dedicated resources.
What I encountered more frequently are attempts to use Python as a full stack language. People are not just using it to implement the scripting frontend interface but also trying to implement the backend that does the heavy processing and the numerical computation. As a developer this is a terrible experience. I'm in the middle of replacing for loops in Python with vectorized numpy code but most of my efficiency gains are erased because I still need Python tuples in the end. Yesterday, I had to consider whether exceptions I throw in Cython code can be caught properly in Python.
Research software engineering is one field where you really do need a full stack language. That kind of software engineering requires high performance with limited resources. Julia with some initial patience does deliver a better developer experience in this scenario than the equivalent Python experience for me partly because I do not need to play vectorizarion games, and I can do everything in one integrated environment.
While, yes, interfaces and static tooling could be better, I do think the situation has gotten better over time. There are interface schemes available and additional static tooling is available.
Julia could deliver a better user experience though. Admittedly, the Python tooling to deploy complex solutions has significantly improved lately via tools such as pixi. Julia tools could deliver generic binary code with packages to ease the initial user experience. Advanced users could re-precompile packages to optimize.
The most promising success I have had with Julia is putting notebook interfaces in front of scientists or deploying web apps. My observation in this scenario is that Julia code can be presented as simple enough for a non-professional programmer to modify themselves. Previously, I have only seen that work well with MATLAB and Python. I have not seen this happen with Rust, and I do not expect that to change.
The other observation is that users appreciate the speed of Julia in two ways. 1. Advanced data visualizations are responsive to changes to data. In Python, I would typically need to precompute or prerender these visualizations. With Julia, this can be done dynamically on the fly. 2. Julia user interfaces respond quickly to user input.
While I think Julia has plenty of room to improve, I do think those improvements are possible. I have also greatly appreciated how much Julia has improved in the past five years.