Have you made or plan to make any contributions to Mezzano (https://github.com/froggey/Mezzano) or are you mainly interested in seeing how far you can take this thing on your own?
I didn't know about Mezzano until now and have never contributed to it. Massive respect to them for what they accomplished. I don't think I have enough knowledge to contribute to a real operating system project like that right now. My experience with Linux drivers is just one small user space driver for my laptop's keyboard LEDs. So for now I think I'll see how far I can take lone.
What’s the minimum kernel version required? (No need for an exact answer, I just want to know if it’s “in the last 3 years” vs “in the last 10 years” etc.)
And is it possible to resolve network names or do anything network related? (Or is it planned?)
I’m always looking for some way to create portable Linux binaries, and I happen to like Lisps. Right now, my best bets are Janet compiled against musl libc or maybe ECL… or just use Python (distributed as source)…
These are the only headers that lone currently requires. When lone is built, a script will read all the system calls defined in those headers and create a table mapping symbols to system call numbers. This is so that you can write symbolic code such as:
(system-call 'write fd buffer length)
Instead of:
(system-call 1 fd buffer length); write is system call 1 on x86_64
Once compiled, however, it should work with literally any version of Linux. The system call primitive can issue any system call given its number and its parameters. If an unsupported system call is made, Linux will simply return -ENOSYS which can be handled at runtime and should not crash the program.
> I’m always looking for some way to create portable Linux binaries, and I happen to like Lisps.
I have this vision in my mind: embedding lone modules into sections of the lone ELF and shipping it out. Zero dependencies, self-contained.
Linux passes processes a pointer to their own ELF header via the auxiliary vector. Lone already grabs this data and puts it into a nice lisp object. Now I just need to code that feature in somehow: parse the header, find the sections, read them into a lone module and evaluate...
SBCL lets you drop core images which if you setup your system properly can be made executable by usesing sbcl as the interpreter, like /bin/sh and shell scripts.
> Lone is a freestanding Lisp interpreter designed to run directly on top of the Linux kernel with full support for Linux system calls. It has zero dependencies, not even the C standard library.
Cool project! Not sure if I'm going to start using it any time soon, but cool nonetheless.
This is interesting. Although, I think it's always useful to point out that runtimes that directly make system calls will have portability issues in other *nix systems. For instance, OpenBSD has recently restricted system calls to a single offset of their libc in order to reduce ROP attacks.
If Lone wishes to be portable, it will need to consider a libc dependency as an option. If not, it can probably get away with direct syscalls on Linux unless Linux kernel developers decide to add support for pinning system calls. I doubt that this would ever be a hard requirement in the kernel, as it would break userland all over, but I could see this being an option for certain distributions as a ROP mitigation. Many features like these flow from OpenBSD to other OSes.
You're absolutely right, it is not portable to other operating systems. I have written about this portability and system call stability here on HN a few times, at some point I decided to compile all the information into a post on my website.
I started lone (and previously a liblinux library) in order to make applications directly targeting the Linux system call binary interface. I chose Linux precisely because it's the only one with a stable interface.
I currently have no plans to make lone a portable programming language.
Many projects start this way. But, as per my comment, the assumption that direct syscall support will be maintained in future Linux distros is also risky.
I think this decision from OpenBSD will more likely discourage developers from even considering it a supportable platform for software that originates in Linux land.
Direct syscall access is not something that is guaranteed in Unix derivatives. Linux is rare in that it provides a stable syscall API. Source compatibility is often only guaranteed when linked against libc or an equivalent low-level runtime library.
So I guess the difference from Janet is no C standard library dependency; is this being targeted for hyper-slim and embedded uses? Because Janet seems like a good choice for most desktop and server cases.
I think it would be really interesting to load the init system (scheme-based GNU Shepherd in this case) directly from the kernel instead of ever loading a shell environment. I bet we could factor out a lot of cruft from the current GNU OS implementation that way, especially if you are managing your system environment/configuration declaratively via scheme/lisp.
Very cool project. Not commenting on that. Lisp and it's derivatives often hits the front page of HN and I always wonder why. What is it about lisp that is so powerful? so much so that some see it as the platonic ideal of programming languages or so it seems?
I have a lot of respect for lisp and its heritage. Respect is certainly one reason why I chose to write a lisp.
Other reasons include simplicity, practicality and ease of implementation. Lisp has a very simple syntax and it is relatively easy to parse it and implement a basic interpreter. I wrote the lexer and the parser by hand, there was no need to mess around with parser generators.
Another reason is I've come to see lisp as something of a frontend for C data structures. I have a byte buffer, encoded text, linked lists, resizable arrays, hash tables... Lisp is the language that binds them all together.
Another reason is that I knew how powerful lisp was despite the simplicity. Despite being a small project, lone is already metaprogrammable with FEXPRs. It turned out I needed exactly one bit in order to give lone macros.
if (function->function.flags.evaluate_arguments) {
arguments = lone_evaluate_all(lone, module, environment, arguments);
}
It just doesn't evaluate the arguments if it's a macro. The function gets the lists that represent the expressions of each argument instead of the final value they compute to. And those lists can be manipulated just like any other list.
I think that was the moment I got the fabled enlightenment they say lisp programmers experience. It just brings a smile to my face.
Well, Y combinator, the domain of HN is intimately related to Lisp, as the founder Paul Graham. That is how I landed in this site to begin, and I assume is so for many around. That could explain it a little bit.
Lisp code is made of lists. And lists are the main data type for lisp programs. So you can naturally produce and transform lisp code using lisp code. Not all lisps allow that, though, but I think that's the main differentiator and it's unique in that aspect.
So it's like generating JS code text from JS, but working with JS code as a text is much more confusing compared to working with lisp code as a list.
Beautiful. Also, after 30+ years of writing C and C++, I learned one thing by casually browsing the source code: you can use a preprocessor macro in an #include statement. Thanks.
I used it to include architecture-specific source code and also a generated C file containing a table of Linux system calls defined by the Linux UAPI lone is compiled against.
The makefile defines those macros by passing flags:
Oh so it's a GCC-specific thing... that explains it. In any case, congratulations on the lisp! It's beautiful code. I wish my current codebase was that beautiful. I mean, it's a lot of us contributing to it so the beauty is a kind of "meeting of the minds" situation... but still :)
PS: My current project implements a lisp... rendered as JSON. Some other angle on beauty...
Yes. It has zero runtime dependencies but development currently assumes GNU tools. The test suite for example is entirely written in bash and uses GNU coreutils. I submitted a patch to coreutils to allow env to set argv[0] of programs specifically so I could use it in my test suite.
Currently lone is a single C source file. It could easily be compiled manually if necessary.
I've started reorganizing the repository though so that's likely to change.
Is there a plan to write a lone compiler so as to eventually have lone bootstrapping/compiling itself so that you no longer have to sully your hands with C?
I understand what you mean. By dependencies I meant user space libraries such as glibc and musl.
The language itself is fully self-contained. It initializes itself with nothing but a static array of bytes.
Could be possible to modify lone to run on bare metal instead. Perhaps by replacing the Linux system call code with BIOS I/O functions and replacing the Linux process entry point code with boot code that initializes the CPU and hardware.
>Could be possible to modify lone to run on bare metal instead. Perhaps by replacing the Linux system call code with BIOS I/O functions and replacing the Linux process entry point code with boot code that initializes the CPU and hardware.
Just don't make the mistake golang made. In most systems, the "stable interface to the kernel" isn't the syscall, but the c library.
Hello everyone! I started this project, happy to answer any questions.
I was hoping to polish it up a bit more so it would be worthy of a Show HN, never thought someone else would submit it here. Really made my day!
Have you made or plan to make any contributions to Mezzano (https://github.com/froggey/Mezzano) or are you mainly interested in seeing how far you can take this thing on your own?
I didn't know about Mezzano until now and have never contributed to it. Massive respect to them for what they accomplished. I don't think I have enough knowledge to contribute to a real operating system project like that right now. My experience with Linux drivers is just one small user space driver for my laptop's keyboard LEDs. So for now I think I'll see how far I can take lone.
What’s the minimum kernel version required? (No need for an exact answer, I just want to know if it’s “in the last 3 years” vs “in the last 10 years” etc.)
And is it possible to resolve network names or do anything network related? (Or is it planned?)
I’m always looking for some way to create portable Linux binaries, and I happen to like Lisps. Right now, my best bets are Janet compiled against musl libc or maybe ECL… or just use Python (distributed as source)…
Edit to add: I love the idea. Great work!
> What’s the minimum kernel version required?
In order to build lone, any version after the Linux UAPI headers split should do.
https://lwn.net/Articles/507794/
https://stackoverflow.com/q/18858190/512904
https://github.com/torvalds/linux/tree/master/include/uapi
These are the only headers that lone currently requires. When lone is built, a script will read all the system calls defined in those headers and create a table mapping symbols to system call numbers. This is so that you can write symbolic code such as:
Instead of:
Once compiled, however, it should work with literally any version of Linux. The system call primitive can issue any system call given its number and its parameters. If an unsupported system call is made, Linux will simply return -ENOSYS which can be handled at runtime and should not crash the program.
> I’m always looking for some way to create portable Linux binaries, and I happen to like Lisps.
I have this vision in my mind: embedding lone modules into sections of the lone ELF and shipping it out. Zero dependencies, self-contained.
Linux passes processes a pointer to their own ELF header via the auxiliary vector. Lone already grabs this data and puts it into a nice lisp object. Now I just need to code that feature in somehow: parse the header, find the sections, read them into a lone module and evaluate...
SBCL lets you drop core images which if you setup your system properly can be made executable by usesing sbcl as the interpreter, like /bin/sh and shell scripts.
2 replies →
> Lone is a freestanding Lisp interpreter designed to run directly on top of the Linux kernel with full support for Linux system calls. It has zero dependencies, not even the C standard library.
Cool project! Not sure if I'm going to start using it any time soon, but cool nonetheless.
Thank you!
This is interesting. Although, I think it's always useful to point out that runtimes that directly make system calls will have portability issues in other *nix systems. For instance, OpenBSD has recently restricted system calls to a single offset of their libc in order to reduce ROP attacks.
If Lone wishes to be portable, it will need to consider a libc dependency as an option. If not, it can probably get away with direct syscalls on Linux unless Linux kernel developers decide to add support for pinning system calls. I doubt that this would ever be a hard requirement in the kernel, as it would break userland all over, but I could see this being an option for certain distributions as a ROP mitigation. Many features like these flow from OpenBSD to other OSes.
You're absolutely right, it is not portable to other operating systems. I have written about this portability and system call stability here on HN a few times, at some point I decided to compile all the information into a post on my website.
https://www.matheusmoreira.com/articles/linux-system-calls
I started lone (and previously a liblinux library) in order to make applications directly targeting the Linux system call binary interface. I chose Linux precisely because it's the only one with a stable interface.
I currently have no plans to make lone a portable programming language.
Considering the title and use case it seems to be intentionally pretty specific to Linux
Many projects start this way. But, as per my comment, the assumption that direct syscall support will be maintained in future Linux distros is also risky.
5 replies →
I think this decision from OpenBSD will more likely discourage developers from even considering it a supportable platform for software that originates in Linux land.
Direct syscall access is not something that is guaranteed in Unix derivatives. Linux is rare in that it provides a stable syscall API. Source compatibility is often only guaranteed when linked against libc or an equivalent low-level runtime library.
10 replies →
What decision from OpenBSD?
Note that Linux is the odd one here. Every other relevant system out there is not like Linux on this.
Very cool! In a similar vein there’s Zuo: https://github.com/racket/zuo
Zuo is interesting because (as far as I know) its primary user is the Racket project itself, for bootstrapping the compiler.
That is correct
So I guess the difference from Janet is no C standard library dependency; is this being targeted for hyper-slim and embedded uses? Because Janet seems like a good choice for most desktop and server cases.
Yes. My current goal is to boot Linux directly into Lisp and bring up the rest of the system from there. Perhaps even create a Lisp user space.
Have you looked into GUIX?
I think it would be really interesting to load the init system (scheme-based GNU Shepherd in this case) directly from the kernel instead of ever loading a shell environment. I bet we could factor out a lot of cruft from the current GNU OS implementation that way, especially if you are managing your system environment/configuration declaratively via scheme/lisp.
2 replies →
I'm new to Lisp - would this then be a modern Lisp machine?
2 replies →
Just compile regular Lisp with cosmopolitan. Then the same binary will run on windows, linux, mac, and BIOS. /s
This has been done with Lua, see: https://github.com/jart/cosmopolitan/issues/61
That reminds me I should go debug the patch I sent to cosmopolitan, it ran on my machine but failed continuous integration...
Very cool project. Not commenting on that. Lisp and it's derivatives often hits the front page of HN and I always wonder why. What is it about lisp that is so powerful? so much so that some see it as the platonic ideal of programming languages or so it seems?
> Ask HN: Why is everyone here so obsessed with Lisp?
https://news.ycombinator.com/item?id=20697493
Ahh okay. Paul Graham and the hackability of the language itself. Got it.
1 reply →
I have a lot of respect for lisp and its heritage. Respect is certainly one reason why I chose to write a lisp.
Other reasons include simplicity, practicality and ease of implementation. Lisp has a very simple syntax and it is relatively easy to parse it and implement a basic interpreter. I wrote the lexer and the parser by hand, there was no need to mess around with parser generators.
Another reason is I've come to see lisp as something of a frontend for C data structures. I have a byte buffer, encoded text, linked lists, resizable arrays, hash tables... Lisp is the language that binds them all together.
Another reason is that I knew how powerful lisp was despite the simplicity. Despite being a small project, lone is already metaprogrammable with FEXPRs. It turned out I needed exactly one bit in order to give lone macros.
It just doesn't evaluate the arguments if it's a macro. The function gets the lists that represent the expressions of each argument instead of the final value they compute to. And those lists can be manipulated just like any other list.
I think that was the moment I got the fabled enlightenment they say lisp programmers experience. It just brings a smile to my face.
Well, Y combinator, the domain of HN is intimately related to Lisp, as the founder Paul Graham. That is how I landed in this site to begin, and I assume is so for many around. That could explain it a little bit.
Lisp code is made of lists. And lists are the main data type for lisp programs. So you can naturally produce and transform lisp code using lisp code. Not all lisps allow that, though, but I think that's the main differentiator and it's unique in that aspect.
So it's like generating JS code text from JS, but working with JS code as a text is much more confusing compared to working with lisp code as a list.
It doesn't hurt that HN is written in LISP.
Beautiful. Also, after 30+ years of writing C and C++, I learned one thing by casually browsing the source code: you can use a preprocessor macro in an #include statement. Thanks.
Yes! GCC calls it computed includes.
https://gcc.gnu.org/onlinedocs/cpp/Computed-Includes.html
I used it to include architecture-specific source code and also a generated C file containing a table of Linux system calls defined by the Linux UAPI lone is compiled against.
The makefile defines those macros by passing flags:
Oh so it's a GCC-specific thing... that explains it. In any case, congratulations on the lisp! It's beautiful code. I wish my current codebase was that beautiful. I mean, it's a lot of us contributing to it so the beauty is a kind of "meeting of the minds" situation... but still :)
PS: My current project implements a lisp... rendered as JSON. Some other angle on beauty...
1 reply →
Props to the author, if the code is as well designed and written as the README, this is looking good.
Great project. I love your style of coding and documenting.
Thank you. I've been a lone programmer from a long time, means a lot to me to read that.
Nice. I can see something like this being used with circle (https://github.com/rsta2/circle) to run bare metal on a Pi, too.
A quick glance shows it uses GNU Make but that's only a build time dependency, I guess.
Yes. It has zero runtime dependencies but development currently assumes GNU tools. The test suite for example is entirely written in bash and uses GNU coreutils. I submitted a patch to coreutils to allow env to set argv[0] of programs specifically so I could use it in my test suite.
Currently lone is a single C source file. It could easily be compiled manually if necessary. I've started reorganizing the repository though so that's likely to change.
>Currently lone is a single C source file.
Is there a plan to write a lone compiler so as to eventually have lone bootstrapping/compiling itself so that you no longer have to sully your hands with C?
3 replies →
All software needs dependencies at build time, saying no dependencies always means runtime.
I'm surprised the Kernel doesn't count here.
7 replies →
>zero dependency
>for Linux
There's your dependency.
I understand what you mean. By dependencies I meant user space libraries such as glibc and musl.
The language itself is fully self-contained. It initializes itself with nothing but a static array of bytes.
Could be possible to modify lone to run on bare metal instead. Perhaps by replacing the Linux system call code with BIOS I/O functions and replacing the Linux process entry point code with boot code that initializes the CPU and hardware.
https://wiki.osdev.org/Printing_to_Screen
That would just shift the dependency from Linux to the firmware though.
>Could be possible to modify lone to run on bare metal instead. Perhaps by replacing the Linux system call code with BIOS I/O functions and replacing the Linux process entry point code with boot code that initializes the CPU and hardware.
Just don't make the mistake golang made. In most systems, the "stable interface to the kernel" isn't the syscall, but the c library.
(re: golang on openbsd)
3 replies →
> Could be possible to modify lone to run on bare metal instead. Perhaps by replacing the Linux system call code with BIOS I/O functions
Certainly not. Calling BIOS interrupts requires the system to be running in real mode, which is incompatible with running 32-bit code.
5 replies →
My project runs in your imagination so there’s no hardware or electrical dependency other than brainwave activity.
[dead]