Zig's Incremental Compilation Internals

9 hours ago (mlugg.co.uk)

Zig's toolchain work is continually impressive. While I still don't plan to write software in it, given that I believe memory safety is table stakes, all of this stuff is very, very good. Before the incremental work, it was the toolchain and cross-compiler work. The toolchain stuff has continually been fantastic. I'm very curious to see what they come up with next!

> Semantic analysis is the most difficult part of the compiler to handle incrementally. Perhaps unsurprisingly then, this is where language design starts to matter a lot: while I am pretty confident that most modern languages could support incremental compilation similar to how we do, certain design decisions can make that much more difficult. Zig has had its design tweaked over the years (sometimes controversially) specifically so that it is easier to support fast incremental compilation.

This is something I wish that we had done with Rust. It is impossible to do all of the things at once, though, and we already had a tremendous amount of things to do. This is also part of the "when do you ship 1.0" tradeoff; for our goals with the language, 2015 was the right moment to launch, but if had a few more years to bake things, maybe we could have made compile times way faster. Software engineering is hard.

  • > I believe memory safety is table stakes

    I'm not sure what that means. Java lets you do many things programs may want to do in a memory-safe way but not everything. Rust lets you do fewer things than Java in a memory-safe way, but more things than Zig. Zig lets you do fewer things in a memory-safe way than Rust, but more than C. So among these four languages we already have four levels of memory safety, none of them is 100%, all of them give up something in exchange for what they offer, and different programmers have different preferences for the compromise they prefer, and even that preference is context-dependent. Which of those less-than-100% memory safety compromises is the table stakes? And given that all of these compromises require something that could be quite substantial, depending on the circumstance, in exchange, and consequently programmers with the highest level of knowledge and expertise choose every one of those four in different situations, to me it seems pretty obvious that none of these is "table stakes".

    • I'll say the same thing I said to you as I said to Andrew, last time he and I talked about this: the way that everyone talks about memory safety (with maybe two exceptions, one okay (go) and one I dislike (fil-c)) is that "memory safe language" is about there being a clear delineation between what is memory safe and what is not, and that the unsafe aspect is a superset. Rust and Java both are memory safe, except where explicitly demarcated as not (unsafe in Rust, JNI or sun.misc.unsafe or whatever in Java). Zig and C have no such separation. When I (and others) talk about wanting memory safety, this is the important aspect of the design. This is what enables the "I know statically that a large part of the code is safe, and I also know where to check if something goes wrong" aspect of things.

      37 replies →

    • > I'm not sure what that means. Java lets you do many things in a memory-safe way but not everything. Rust lets you do fewer things than Java in a memory-safe way, but more things than Zig.

      I don't think I agree with this framing. The question to me isn't "what can you do while being memory safe", it's "can you accidentally do something memory unsafe without noticing?" Rust and Java are the same here; you need to explicitly opt into using the language's mechanism for relaxing restrictions (Rust's `unsafe` blocks, Java's `Unsafe` class APIs), whereas from what I understand, neither Zig or C offers anything strict in that way.

      17 replies →

    • safe Rust is actually more memory safe than Java, since it guards against data races (in Java data races are not UB, but they are still one of the worst kinds of bugs because it leads to logically impossible program states)

      Also note that Java has unsafe, but doesn't have the culture of plainly stating safety invariants like Rust. The unsafe features of Java are less widely used, but when they are you rarely know if a Java library has unsafe internals for performance, and if they do, it may be hard to audit

    • Java and Rust have actually very similar memory safety profiles. Rust let's you within the language escape the memory safety requirements whereas Java does not but both are considered memory safe languages. Rust also enforces thread safety as well which Java does not, but the slower JVM memory model doesn't let race conditions become memory safety issues whereas Rust is lower-level like Zig/C/C++ and thus thread safety could be a memory safety issue.

      Zig has an identical memory safety profile to C. It has facilities to make it easier to stay memory safe, but those facilities are basically equivalent to what you have in C++ and that's equivalent memory safety profile as C.

      > So among these four languages we already have four levels of memory safety, none of them is 100%

      No, you've pretended like there's four when really it's Java / Rust which are safe by default and Zig/C/C++ which are unsafe by default.

      One effective metric to evaluate is memory safety per LoC. Rust is ~0.2 vulnerabilities per MLoC. Java is effectively 0. C and C++ both seem to be about 1,000 vulnerabilities per MLoC. Zig is too new and hasn't had any analysis done on it, but generously it's likely at least 10-100.

      So the table stakes could be defined as 1 memory safety vulnerability per MLoC.

      12 replies →

  • 10 years ago, I commented on the Rust issue for "Incremental recompilation", where it was suggested that Rust could at least adopt Haskell GHC's model of incrementality, which is currently file-level:

    https://github.com/rust-lang/rust/issues/2369#issuecomment-1...

    This would already help a lot.

    I recommend anybody who's interested in incremental recompilation to read what GHC does, because the effort to achieve that is relatively low.

    Of course there's always desire for more:

    GHC currently needs to parse+typecheck+codegen a file before it can process other files that import it. Codegen is slow. Thus, there's currently demand split compilation into "stages", so that the next file can be typechecked after its imports have been just typechecked (not codegenned).

    I would also enjoy if recompilation avoidance were to happen at the function level, not the file level.

    Macro systems are a key language feature that can destroy incremental recompilation. In theory, Haskell is well set up for that, as its macro system (TemplateHaskell) is fully AST based and _theoretically_ could distinguish "fully pure" macros from side-effectful macros (such as splicing the current git commit in as a string literal). But the recompilation avoidance system does not currently exploit such differences.

    • Can't we have a system where we trade some performance for quick incremental compilation?

      We can always compile with full optimization just before shipping?

      2 replies →

  • The thing with rust is that you get safety with slow compilation, it's a tradeoff.

    Zig doesn't have the same safety guarantees, it's on the dev to use safe coding patterns, so the tradeoff for safety is discipline or experience.

    • Rust's safety checks have basically nothing to do with its slow compile times. This is something that sounds intuitive but is just completely incorrect.

      In particular, Rust made several good design decisions around this stuff that keeps those checks fast, like keeping checks local rather than being global.

      19 replies →

    • (tongue in cheek) It seems recent history has shown that zig can get you to working software faster, then you can port it to rust once you are acquired or find market fit?

      Maybe at some point in the future zig could add a rust compilation target ( like with `-ofmt=c` )...

      2 replies →

This post is really interesting. As a member of the rust-analyzer team, I cannot avoid comparing it to the situation in Rust land. Rust famously has not less (or even more) sophisticated system for incremental compilation, yet its compilation is way slower. I attribute that to two main things:

- Language design. Zig was designed for fast and incremental compilation, Rust is just not. For instance, the post states that Zig has four properties (layout, type, value, body) that the compiler has to track for changes. Rust has much more, to the point that tracking them statically is just impossible, so the compiler uses a query system that tracks them dynamically, which adds overhead.

- Compiler implementation. Rust is much more complicated to compile than Zig, and rustc is both older and bigger (10x-20x LOC) than the Zig compiler, making changing it way harder.

I'm becoming a big fan of Zig every since learning about `zig cc` as a way of dipping my toes in it. I was already impressed by the build caching so I'm keen to play with this

There is something that I don't fully understand about this design: why are they insisting on building a giant binary for debug builds that contains all of the code? From my perspective, a simpler approach is to generate many smaller shared libraries (perhaps at the file level) and link them in to the final binary. With this approach, the program binary would have a tiny text section and a (potentially long) list of shared libraries to load. But even with thousands of shared libraries to load, the resulting program binary would not be all that long and there would be no need for binary patching.

I understand that for a release mode a single giant binary may be desirable, but I am struggling to understand this design for debug builds. Moreover, while reading this article, I found myself wondering what happens if the main binary becomes corrupted. Maybe the user cancels compilation with ctrl+c while it is patching the binary. Even if they have a story for avoiding and/or detecting corruption, it is simpler to not patch in the first place and always generate a new main binary. Again, this is reasonable because the new binary is mainly just a list of shared libraries to link which will not take up much space and can be written to disk quickly. Moreover, this process can be done recursively, e.g. at the subdirectory level, so that during incremental linking a few quite small shared libraries may be produced rather than patching in the new code and writing cascading relocations.

  • A quick test (C, clang) gives me that a binary depending on 1000 shared libraries, each containing a single function returning an integer, with a main function summing up the results of all those functions, takes ~270ms to run from the dynamic linker overhead. So you'd definitely want a good chunk below thousands.

    (a process with 100 shared libraries takes 6ms to run, which is a lot better (0.9ms for 1 library, for reference), but, especially with in-place patching skipping work on unchanged values, static linking still has a good shot at beating dynamic linking, especially if you run the binary multiple times)

    Incremental compilation generally already depends on its stored intermediate data not getting corrupted, and the final binary need not be any differently handled in that aspect.

  • If you choose to build static or dynamic libraries, I expect you’d still get to do that, and I expect as long as those don’t change and only the main binary needs patching it’d work the same. (Though I’m waiting for the next tagged release to really try it out myself, so that’s not like a guarantee or anything.)

> Dependencies on the body of a runtime function are impossible (at least in the simplified view I’m presenting here)

How does this work given that e.g. a constant can be computed by a comptime function?

  • It works through the fact that I specified "runtime" function ;)

    A bit after that quote I have a note about `inline` functions in Zig, where I mention that they perform semantic inlining, which means dependencies triggered by the function actually get associated with the call site. Well, `comptime` function calls work just the same way---in fact, to the compiler, `comptime` calls are almost exactly identical to `inline` calls. So when we encounter a comptime function call, we start analyzing the ZIR for that function's body, but we don't switch our analysis unit, so comptime stuff doesn't really complicate the dependency graph at all (aside from the fact that it means you can depend on any number of source code hashes, instead of everything depending on exactly one).

    With all that being said, there actually is a (completely unrelated) way in Zig to can depend on the body of a runtime function (hence why the quote includes "at least in the simplified view I'm presenting here"). It's to do with "inferred error sets" (IESes for short). If a function's return type is written `!T`, that means it can return an error, but we're asking the compiler to figure out exactly which errors are possible. So if at some point we need to know that set of errors (e.g. because the user has done some reflection to try and access the list of errors), that's where we get a dependency on a runtime function body, because we need to analyze the function body to learn about all the places it might return an error.

I've always thought this was fascinating, but the only incremental compilation I knew was obscure programming languages and Rust. Oh yeah, I guess Roslyn?

Really fun and fascinating problem to work on.

  • Wasnt Roslyn 1st at implementing this on such scale?

    • Yeah I remember being in sophomore year of college, watching Anders Hejlsberg's video on "the new way to build compilers" or something and having my mind blown. But I only ever looked at the source code for Rust when it came to something actually implementing this, so that came to mind first.

      Roslyn also has an extra constraint of integrating with live editing on the fly; I think you can get simpler and/or have different constraints if your requirement is only incremental compilation.

The zig compiler can compile C so will it work with C also?

  • not quite! if your project is mixed C/Zig, then editing Zig would work but editing C would not. the Zig compiler caches Zig AIR but no information from C. plus, the C compilation can only really be done with LLVM and writing a self-hosted backend would be PITA for C

    • Not quite true, there is already a capable C compiler written in Zig (Aro/arocc), and a plan to transition to it for C compilation: https://codeberg.org/ziglang/translate-c

      Using this native (written in Zig) C compiler to translate C source into Zig source as a part of the build, would presumably lend itself trivially to all the incremental logic in TFA, as updating C would update the generated Zig, and the incremental logic would detect differences just like it detects differences made by a human in an editor. Maybe there are aspects of the generated Zig that would complicate that somewhat, but I don't know -- just a warning about my ignorance.

      This is part of plans to remove the hard LLVM dependency. AFAIK, the LLVM dependency will still be a variant many will use for the convenience of Zig as a much better clang, but removing the hard dependency is part of enabling all these great features like incremental compilation.

      4 replies →

Does this work for release builds or just debug builds now?

  • It only works with our self-hosted code generation backends (the main one being for x86_64), which right now don't have any optimisation passes. It's planned that they will in future, but that's a long-term goal. Also, any optimisations which propagate information between functions (the most obvious and important one is inlining) are more-or-less incompatible with incremental compilation (I touch on this in the post IIRC), so once this does work it'll probably still be limited to a subset of optimisations.

    TL;DR: only debug builds for now, could extend to release builds once we have our own optimisation passes one day, but some optimisations will still be inapplicable.

  • I don't think it will work with llvm/release yet but it might some day maybe.

    Getting incremental linker to work with llvm is kinda hard atm. Maybe the zig team has a plan dunno.

It disappoints me how unseriously the industry has taken compilation speed for so long. I'm glad to see Zig doing incredibly valuable, high-impact work here.

  • Well, I remember back in the day Java was winning a lot of love from C++ devs over its much-much faster compilation than C++. It was important, even 15+ years ago. People praised faster iteration time while coding.

  • Yes, we get people amazed to rediscover Modula-2, Object Pascal compiler execution speed.

The incremental linking part sounds pretty hackish to me and I wonder what the price is in increased code complexity and maintenance effort. It also does not mention how it deals with the patching of debug information.

I just looked up a Hello World program from the Zig Wikipedia article:

    const std = @import("std");
    const File = std.Io.File;
    
    pub fn main(init: std.process.Init) !void {
        _ = try File.stdout().writeStreamingAll(init.io, "Hello, World!\n");
    }

That's a lot to follow, just to output a plan-text message, especially after this line: "The primary goal of Zig is to be a better solution to the sorts of tasks that are currently solved with C. A primary concern in that respect is readability…"

  • I don’t think you can judge a programming language based on its “Hello World.” AppleSoft BASIC has this:

    10 PRINT “Hello World”

    Beautifully simple and readable. But it’s not a good language by modern standards.

    In your example, I see a lot of complexity being surfaced: output streams, locals instead of globals, error handling. I don’t know Zig but all of those are things that are important to address, and I like that the example doesn’t sweep them under the rug in pursuit of a false readability.

  • Everything it's doing it clear and readable. It's just not as easy to write. It streams the bytes to stdout using the default IO interface, and it can fail.

    Alternatively, here's a simpler version (prints to stderr).

        const std = @import("std");
    
        pub fn main() void {
            std.debug.print("Hello, world!\n", .{}); 
        }
    

    In practice, you normally don't want to print messages to stdout. So the increased friction here actually pushes you in a better direction.

  • The issue is that most "hello world" programs are not correct.

    • While most hello worlds do not check that the message was printed (which I assume writeStreamingAll does for you), dismissing the rest of the differences as "the others aren't correct" isn't really accurate.

      Explicitly passing IO in is a fine design choice, but it's not a correctness issue to say others are wrong to not do so.

      2 replies →