Comment by norir
2 days ago
Having scanned the source, I believe there is a better design for this language.
There exists a much smaller subset of jank that is capable of compiling itself. I am quite sure that you could write a bootstrap compiler in clojure (but restricted to the same subset) that translates to clojure (or some other language you prefer targeting). My guess, from experience writing small compilers, is it should be doable in ~5000 lines or even less.
Once you had that, you could then write a backend that translates to c. This gives you a native compiler for this small subset of jank. Applied to its own source code, you get a native compiler binary for free and cut the cord from clojure.
Now you have a native compiler and a very small, tight language. Because it is so small, it is easy both to check correctness and optimize. You can add new backends relatively easily. For the repl, you can compile functions into their own hash-addressed shared libraries using a c compiler and dlopen them. TCC is very fast for this purpose even if the generated code isn't quite as good as optimized gcc or clang.
All of the future vision could be implemented on top of this foundation. The current approach is likely going to just bog the author down, dealing with accidental complexity from c++ and llvm especially. It is also better to have fewer features implemented completely and correctly than more features incompletely or incorrectly implemented.
> There exists a much smaller subset of jank that is capable of compiling itself.
If your fitness test is compiler SLOC, there does indeed exist better designs. However, each of those will sacrifice performance and robustness in favor of that simplicity and concision.
I don't want to implement jank in jank. I don't think jank will ever be self-hosting. Why? Multiple reasons.
Firstly, jank isn't a systems language and in order to perform the tricks necessary for a fast runtime, I need a systems language like C++. I want jank to be the fastest Clojure around and that will require more than just a 5000 line small self-hosted compiler.
Secondly, jank has two audiences. The first audience is existing Clojure devs. A self-hosted jank would work well for them. However, the second, larger, audience is existing native devs. For them, jank needs to have an idiomatic native interface in order to embed it into their programs. The Clojure side of jank is not designed to create C++ APIs; it's designed to consume them. However, the C++ side of jank _is_ a C++ API and is designed to be a library.
Thirdly, jank's seamless interop is unprecedented for a lisp. This requires the bleeding edge (literally unreleased LLVM main, as of writing) of Clang and LLVM in order to be possible. This is necessary both for full JIT capabilities as well as the ability to seamlessly reach into C++ to use any value, function, or type, from jank, even when that requires JIT template instantiation. The typical small compiler approach of compiling to C and using TCC will not allow me to achieve this. Again, different fitness test.
> The current approach is likely going to just bog the author down, dealing with accidental complexity from c++ and llvm especially.
It sounds to me like you're designing another language, not jank. With all of this said, of course jank's design and code could be improved. I'd be able to point out more issues in jank's code than anyone, I'd wager, since I've introduced most of them. But the bulk of your point, of having less C++ and more jank, would lead to a smaller, slower, and less robust language. I will pay the cost of C++ in order to build the language I want.