Comment by ninkendo
3 days ago
> Incremental compilation is not cleaned up. Older deps pile up and don’t get cleaned.
This is one of those problems that’s both huge/important, and probably impossible to solve. It bites me all the time, I have a measly 1TB nvme disk and I basically have to wipe my target directory and docker build cache (I have a shared cargo cache volume across builds) every other day. I’ve had this workstation for 2.5 years now and I’m honestly worried about nand endurance coming to bite me soon.
How would one even solve this? When do you know a cache entry won’t be used any more? Trace back provenance of artifacts to their sources, and if the mtimes have been bumped just delete the artifacts? It’s not like resetting source files to older mtimes is a good idea anyway…
I disagree it’s impossible to solve. Huge C++ projects don’t have continuously growing build directories and can still compile incrementally correctly.
That’s because they overwrite the old artifacts with new ones every time, which has its own problems: you could end up reusing stale artifacts if your compiler version changes, or if different compiler args are used for the same artifact, and a bunch of other stuff. Rust computes a digest of all the relevant build environment settings that may need a different artifact to be built, and uses that digest in the file name. That means you basically never have issues like “oh crap the incremental build broke, do a clean build instead”. (Amongst a whole lot of other benefits I’m sure.)
But this technique errs on the side of not using invalid artifacts, at the cost of not having a good system for expiring them. To maintain the goal of never having to worry about when to require a clean rebuild, solving cleaning of stale artifacts becomes a much harder problem.