Fusion – A hobby OS implemented in Nim

2 years ago (github.com)

Author here. Thanks for posting this <3 Happy to answer questions.

  • Congrats, fascinating work!

    Q: has the GC of Nim caused any challenges?

    (And if not, would you attribute that to Nim unique GC that does NOT “stop-the-world”?)

    https://nim-lang.org/1.4.0/gc.html

    • Thanks! As @Tiberium mentioned, Nim 2.0 defaults to using ARC (Automatic Reference Counting), so no runtime GC. The Nim compiler is quite smart about when to copy/move and when to destruct, with some hints like lent/sink for when you want to have a bit more control.

      Keep in mind that I also need to use ptr (as opposed to ref) types in a lot of cases when working at a low level, so there might be a need for some manual memory management as well.

  • Amazing work! What where the most troublesome parts of the project? Also, any tips if anyone want to write an OS from scratch, aswell?

    • > What where the most troublesome parts of the project?

      Task switching. It's a very intricate process, you really have to understand how to structure everything (especially the task stack) so that you can switch everything from underneath the CPU as if nothing has happened (the CPU is obliviuous to which task is running, including the kernel itself). Add to that switching between user mode and kernel mode (during interrupts and system calls) and it becomes even more challenging.

      > Also, any tips if anyone want to write an OS from scratch, aswell?

      As @deaddod said, you need to read a lot. Two invaluable resources for me were the Intel Software Development Manuals (SDM), and the osdev wiki. The SDM can be daunting, but it's surprisingly very readable. The osdev wiki has great content, but it can be a hit or miss. I complement them with various blog posts and code on github to really understand a certain topic.

      That being said, the most important aspect of this process is to have tons of curiousity and to be passionate about low-level systems programming. I love to learn how things really work at the lowest level. Once you learn it, you'll discover that there's no magic, and that you can write something yourself to make it work.

      [0] https://www.intel.com/content/www/us/en/developer/articles/t...

      [1] https://wiki.osdev.org

    • Read. A lot.

      Basically, OSDev is one of the few realms you can't really take shortcuts in. It's kind of like learning Rust, in that you'll have a lot of foundational work until you get some real payoff. Unlike rust, however, there isn't just some cliff where you start getting it; it's a constant uphill trudge.

      Learning the boot process of your target architecture, adding core functionality (process scheduling, filesystem/VFS support, IO management, etc), adding driver support, supporting your video device (just getting a basic framebuffer, and then adding each piece after that), supporting USB, supporting CRT and POSIX (if you choose to do so), etc are all herculean tasks of their own.

      That being said, it's a super incremental process, so you'll get to watch it grow and expand at each step.

      Reading up on the FreeBSD and Linux kernels are good starts. As well as reviewing other hobby OSes such as Serenity, TouruOS, Haiku, etc. And the OSDev wiki is invaluable.

      Also, accepting you probably aren't going to build the next big OS or trying to compete with the big dogs is something you'll have to humble yourself with.

  • I don't use Nim, but it's an interesting language. I've read someone else complaining about having to make changes to an old project every time he went to recompile it. I'm wondering how true this is, so in other words, I'm wondering about the frequency and severity of breaking changes in the language.

    • I haven't faced such issue. I think the only issue I faced was when I upgraded to 2.0, they made `--threads:on` the default, so I had to turn it off, but that's about it.

    • Doesn't happen a lot, but it does happen from time to time. Of course only if you actually update your Nim version, so it's not like interpreted languages like Python where stuff stops working if a new version comes out and your package manager upgrades it.

  • Q: what made you choose Nim over Swift?

    (since they seem very similar at this point, yet Swift is more battle tested)

    • I did consider Swift for a brief moment, but was put off by the fact that targeting bare-metal was almost non-existent. It was more work than I would like to put into bootstrapping the dev environment.

      9 replies →

  • I heard that null is a valid value for objects in Nim for most situations. Is that correct ?

    I like languages that disallow null by default (e.g. Rust, OCaml etc) because it seems to be a huge source of errors.

    • No, it isn't. The type must explicitly be a nullable type or pointer to be nullable. All other values must have a valid initialization. Anything not marked as a pointer or ref is by default managed by its scope. This includes dynamic types like seqs and strings, which are pointers to heap memory but managed by the stack scope and deallocated upon leaving scope.

    • To be honest, I haven't found this to be an issue (yet). I try to keep most of my types value types (cannot be null), which the compiler can pass by reference under the hood if it detects that it's too big, without compromising memory safety.

    • I use the Options module which has a none/some check. None is the absence of a value. You can test for this quite easily and I see it as a feature, not a bug.

Your blog/docs are excellent. Perfect balance of showing and telling. Thanks so much for taking the time to share what you're doing like this.

  • Thanks! Actually writing has helped me so many times in improving the design and implementation. It forces me to question my assumptions, and ask myself: would the reader be able to understand why I made such decision? I have to justify everything I do, which helped me remove unnecessary complexity and focus on the more important aspects.

Nice, I love to see stuff like this. I've been an on-again, off-again Nim "ecosystem guy" for several years. It's great to see this delightful little project is still chugging along.

Nifty! Fun to pull up the module for ELF and have it be so easy to read.

Some day I want to write an RTOS in Nim. I enjoy writing embedded programs in Nim and it’d be fun to make an RTOS.

  • That would be great! I'd love to follow along if you ever decide to build an RTOS.

    • To either of you, whenever you are doing something new from scratch, it can be useful to consider the granularity of provided abstractions & services which khaledh seems to be doing. I see fusion only has like 8 syscalls presently. It's not in Nim, but along these lines ("how much" individual calls do), you might want to consider an approach like this: https://github.com/c-blake/batch to amortize costs of crossing expensive call boundaries.

      4 replies →

What is Nim, and what is the overarching design goal for Fusion? Thanks.

I'm hoping these questions aren't too basic, I have no context whatsoever for understanding this so hope someone can explain.

  • As others mentioned, Nim is a statically typed programming language that compiles down to C, C++, and JavaScript. It has great C interop, which makes systems programming easy. As for why Nim, here's an excerpt from my accompanying site[0]:

    > Why Nim? It's one of the few languages that allow low-level systems programming with deterministic memory management (garbage collector is optional) with destructors and move semantics. It's also statically typed, which provides greater type safety. It also supports inline assembly, which is a must for OS development. Other options include C, C++, Rust, and Zig. They're great languages, but I chose Nim for its simplicity, elegance, and performance.

    As for the overall design goals of Fusion, I have high ambitions, which I list on the same page I referenced. I don't want to build another Unix-like OS; I'd like to experiment with fundamental issues in OS design, such as using a single-address space and capability-based security for protection. Another aspect I'm trying to explore is how processes/tasks are modeled, which I believe should be modeled as state machines with statically-typed channels to communicate between each other (this is not new, it's been done in Singularity OS[1]). There's rudimentary support in the kernel for channels and using them from user space, but it's still early.

    [0] https://0xc0ffee.netlify.app/osdev/01-intro.html

    [1] https://en.wikipedia.org/wiki/Singularity_(operating_system)

Seeing more projects in Nim makes me happy. I'm a (mostly) Python and JavaScript programmer who is interested in the benefits of also knowing a modern, fast, statically-typed language. Among a candidate list of Go, Rust, Zig, or Nim, I like Nim the most. It feels the most "Pythonic" in the sense of very little syntax clutter when I'm reading code. I also love, love, love using a REPL to prototype new code, and INim does it well. The biggest problem with Nim currently is its small community size, which makes the universe of available and maintained software libraries smaller than in the other language communities. It's a chicken-or-egg problem, but can be solved by more devs (including me!) being "the change you want to see in the world".

  • To be honest, I haven't found the community size to be an issue. The Nim forum[0] has a vibrant community, and is the place I go to for help, and the response is usually quick and on point. The language is also evolving in a careful manner, with Araq at the helm I think it's going to be even better in the long term.

    As for the ecosystem, yes, it's not as big as Python or Rust, but surprisingly the standard library has most of what people need. I rarely look for 3rd party packages to do something.

    That being said, I acknowledge that Nim is on the lesser known languages of the spectrum, but that doesn't take away from its merits as a very promising language that does what it's supposed to do very well.

    One thing I think the community should focus on more is IDE support. The VSCode extension is good, but has some rough edges. I also prefer JetBrains IDEs, and the official Nim plugin is very lacking to say the least. I have another side project to create a JetBrains plugin for Nim[1], but I haven't gone far with it yet.

    [0] https://forum.nim-lang.org

    [1] https://github.com/khaledh/nimjet

    • My world: I need to use OpenCV. The existing OpenCV bindings (nim-opencv) haven't been touched in years because the author left* the Nim community. (And that really stinks! It was created by dom96, who also created Nimble, Jester, and a ton of other useful stuff in the Nim world.) ... So... I created my own OpenCV bindings and published it (https://nimble.directory/pkg/mvb). But they're minimal because I'm just one dude and haven't had the time to complete the bindings (either manually, or ideally, using an automated binding generator tool). I will.. eventually.. I hope! Meanwhile, OpenCV bindings for Rust and Go are robust and well-maintained.

      Now I'm playing around more with Nostr (and the Lightning Network)... Nostr libraries for Nim are not as complete or well-maintained as those in Rust, Go, or even Python, etc.

      I'm not letting that stop me from using Nim for my projects... I love Nim! But it does mean I have more work to do (and code to maintain). I can make that choice because I'm my own boss and run my own company. But I could see others not making the same choice for rational reasons.

      * And dom96 left, unfortunately, because of harassment and abuse, which is another possible reason why Nim isn't as well adopted as Go, Rust, etc. If people want to see Nim succeed more, they also need to focus on improving community safety, too. https://news.ycombinator.com/item?id=38999296

      2 replies →

Nim is a great systems language, and should be more popular.

  • Maybe it's just because I'm getting older, but I struggle to read Nim when I'm forced to use two spaces for indentation. I can barely make out each block of code, and I don't like to rely on my IDE to make a language readable.

    • I use it at work for an important CLI tool that backs a number of our systems. My style is almost exactly like Python. Four-space tabs, snake_case, my own rules for indenting parameters and keywords. You do not need to use the official Nim style, you can absolutely write it like Python, Ada, however you like.

      2 replies →

The problem that nim have and many afraid to go nim is case and style insensitivity.

Is_land == island == IsLaND == is-land

It is bad in team setting, in real world projects.

How it goes now ? Last time I checked the main dev refuse to do anything about against popularity vote In Github.

Otherwise awesome project and documentation Fusion Os

  • It is so that you can use external libraries with your preferred style without having to convert.

    It is a pretty amazing feature. Your problem is just imaginary. A consistent case style should always be enforced regardless if you have a case insensitive language or not. There is no real world case where you would want is_land and isLand to exists both in your code and be separate variables.

    • There are many code that have `class restController` in title cases and instance `restcontroller = RestController()` you can check around in the GitHub world.

      How it would effect? I haven't touched nim since that decision to keep those insensitivities.

      1 reply →

  • I agree that it's unusual (and likely scares off some), however it's not entirely case insensitive. First, dashes/hyphens (`-`) can't be part of identifiers. Second, the first character of an identifier is not case insensitive.

    So:

    FooBar != fooBar

    FooBar == Foobar

    Most of the developers in the community are ambivalent about it, because it rarely ever causes problems. If you end up misspelling an identifier, you're nearly always going to get a compile-time error due to static typing anyway.

    • Only the first letter being case-sensitive is a major strike against readability, one of four major pillars. While I’m sure the Nim developers are probably used to it by now, it just seems like a bad design decision Nim is probably burdened with as the result of legacy/interoperabilty.

      Even just reading your foobar example at a glance took a moment for me.

      And case insensitivity is also generally frowned upon. To have a language with both sensitivity and insensitivity is the worst of all worlds with none of the benefits.

      If you want to understand why at a deeper level I would recommend reading readability or the case insensitivity sections in any programming languages book. Personally, I enjoy Programming Languages, Principles and Practice (Louden & Lambert)

      EDIT: Yes, I get it, it doesn't affect YOU. But it doesn't mean it doesn't affect other people. Non-english languages and/or speakers are an easy example. It also eliminates a whole class of human error, and maybe that only affects non-experienced juniors, but they exist too. There are other issues with symbols being case insensitive and string values being case sensitive. If you want a practical example a classic one is HttpsFtpConn vs. HttpSftpConn

      4 replies →

    • You can also enable --styleCheck:usages, which warns about casing inconsistencies in your code. (So it catches mistyping FooBar as FooBAr.) Then the only difference from other languages remains that you can't use weird casing differences for separate symbols, e.g. you can't name two separate variables fooBar and foo_bar, which you wouldn't normally do anyway.

    • Ha this always happens with case insensitivity. It's case insensitive.... except for some situations you need to now remember. I believe PHP has this issue too.

      It isn't a good look that they made the same mistakes as PHP.

      7 replies →

    • That's is even more unexpected.

      FooBar != fooBar

      FooBar == Foobar

      That could cause a lot of head ache in large code base ..

  • I've seen Delphi ERP projects worked on by dozens of people and case insensitivty of Pascal was never ever the issue. You choose something and stick to it. The concerns and fears are mostly due to inexperience.

    • True, but in Delphi's case _ and - aren't part of case insensitivty, that is probably a bit too far.

    • And no such thing ever exist in other languages as first letter sensitive identifiers

  • Ends up being a total non-issue in the real world. I use it for a mission-critical CLI application called by a number of backend systems. I use snake_case for virtually everything save for object type names. There is no global scope and modules keep most symbols segregated. I have yet to have a symbol collision issue. I get to use foreign package symbols in my own style. On multi-dev projects I'd expect a style to be enforced, and given module segregation, a collision should be fairly simple to suss out. Especially if you use `from module import nil` and force module name qualification.

  • I think this problem can be solved using either a linter or formatter-like tool that makes naming consistent before the code gets committed.