← Back to context

Comment by maccard

2 days ago

I work on large c++ code bases day in day out - think 30 minute compiles on an i9 with 128GB ram and NVMe drives.

Rusts compile times are still ungodly slow. I contributed to a “small to medium” open source project [0] a while back, fixing a few issues that we came across when using it. Given that the project is approximately 3 orders of magnitude smaller than my day to day project, a clean build of a few thousand lines of rust took close to 10 minutes. Incremental changes to the project were still closer to a minute at the time. I’ve never worked on a 5m+ LOC project in rust, but I can only imagine how long it would take.

On the flip side, I also submitted some patches to a golang program of a similar size [1] and it was faster to clone, install dependencies and clean build that project than a single file change to the rust project was.

[0] https://github.com/getsentry/symbolicator

[1] https://github.com/buildkite/agent

Thanks for actually including the slow repo in your comment. My results on a Ryzen 5900X:

* Clean debug build: 1m 22s

* Incremental debug build: 13s

* Clean release build: 1m 51s

* Incremental release build: 24s

Incremental builds were done by changing one line in creates/symbolicator/src/cli.rs.

It's not great, but it sounds like your experience was much worse for some reason.

  • For reference, buildkite-agent [0] is about 40k lines of go. Running `go build` including dependencies took 40 seconds, and running `go clean && go build` took 2 seconds. I know Go and Rust aren't comparable, but Rust's attitude appears to be "we don't really care" when you compare it to Go, considering they both started at _roughly_ the same time and Rust's first stable release came long after Go was in use.

    [0] https://github.com/buildkite/agent

  • Sorry - my clean build was actually including the dependency fetching, which is a large part of it. My experience was in 2023 which if we go by article roughly scales with compiler performance to 5 minutes or so

    • I do wonder how much the fetch-source-and-build-all-dependencies approach contributes to the poor perception of compile times -- your very first experience of "how long did cargo take to build this project" will be terrible because it will include downloading and building a ton of dependencies...

      1 reply →

    • My build times also included fetching dependencies, which only took a few seconds on my modestly fast 8 MB/s network.

> clean build of a few thousand lines of rust took close to 10 minutes

That doesn't sound likely. I would expect seconds unless something very odd is happing.

Is the example symbolicator?

I can't build the optional "symbolicator-crash" crate because it's not rust but 300k of C++/C pulled from a git submodule that requires dependencies I am not going to install. Your complaint might literally be about C++!

For the rest of the workspace, 60k of rust builds in 60 seconds

- clean debug build on a 6 year old 3900X (which is under load because I am working)

- time includes fetching 650 deps over a poor network and building them (the real line count of the build is likely 100s of thousands or millions of lines of code)

- subsequent release build took 100s

- I use the mold linker which is advised for faster builds

- modern cpus are so much faster than my machine they might not even take 10s

  • > That doesn't sound likely. I would expect seconds unless something very odd is happing.

    And yet here we are.

    There are plenty of stories like this floating around of degenerate cases of small projects. Here's [0] one example with numbers and how they solved it. There are enough of these issues that by getting bogged down in "well technically it's not Rust's fault, it's LLVM's single threadedness causing the slowdown here" ignores the point - Rust (very fairly) has a rep for being dog slow to compile even compared to large C++ projects

    > For the rest of the workspace, 60k of rust builds in 60 seconds

    That's... not fast.

    https://github.com/buildkite/agent is 40k lines of go according to cloc, and running `go build` including pulling dependencies takes 40 seconds. Without pulling dependencies it's 2 seconds. _That's_ fast.

    [0] https://www.feldera.com/blog/cutting-down-rust-compile-times...

Just curious, are you still able to get instant feedback and development conveniences on that 30 minute compile time project, like up to date autocomplete and type hints and real-time errors/warnings while developing before compiling?

  • Yeah - there’s a 60-ish second delay in my IDE before this info is available but once it’s there it’s there.

Can you say what your development environment was like? I was having 15 minute build times for a pretty small system. Everyone talks about how slow Rust compile times are so I thought that's just how it is. Then, by chance, I ended up building from a clean install on my work laptop and it took about 3 minutes from scratch.

My development environment is VS Code running in a Dev container in docker desktop. So after my work laptop was so fast, I made some changes to my Mac docker desktop and suddenly the mac could build the project from scratch in about 2 minutes. Incremental compile was several minutes before, instant now.

  • Cargo in vscode on windows on a monstrously big machine (3990x/128GB RAM/NVMe drive, Gigabit Ethernet)

    I think if it's that sensitive to environment issues, that solidifies the point that there are major problems that lots of people are going to have.

Yes, but Go is a higher level language than Rust. It feels unfair to compare the two. That's why I brought up C++ (as did the article).

  • I disagree that it’s unfair to compare the two. The performance difference between go and rust is far less than the difference between go and python, it has a garbage collector sure but it’s an example of a language designed for fast compilation time that achieves an order of magnitude faster compile times than rust

30 minutes versus 60 is really an hour versus two.

Some coworkers and I noticed a long time ago that once you try to task switch while doing build/test automation steps, it always seems like you remember to come back and check about twice as long as the compile was supposed to take. 7+ turned into 15, 15 into a half hour.

And then one day it hit me that this is just Hofstadter’s Law. You think you have ten minutes so you start a ten minute task and it takes you twenty, or you get in a flow and forget to look until your senses tell you you’re forgetting something.

Cutting 10 minutes off a build really averages 20 minutes in saved time per cycle. Which matters a hell of a lot when you go from 4 to 5 cycles per 8 hour day.

What are incremental compile times with the C++ codebase?

Also, does the line of code you count include dependencies (admitting, dependencies in Rust are a problem, but it's not related to compiler performance)?