There’s an interesting comment thread (including potential downsides) about this technique in the submission for “How to speed up the Rust compiler in September 2026” from a couple days ago: https://news.ycombinator.com/item?id=49923594
Huh, I kinda thought something like this was already being done, but perhaps it was a bit later stage. Neat!
A kind of related shower thought I had, could rustc not instantiate generic methods, but write out what it expects to find (std::Vec<String>::push), and a separate build process listens to these, but makes sure already built ones are not built multiple times from the whole build.
I don’t actually know how much of duplication there usually is, but I had gotten the impression that it would be part of the problem.
Of course I’m no compiler engineer, and I’m sure there are complications like per crate build profiles, etc.
> I kinda thought something like this was already being done
It is. When building a crate, Cargo doesn't need to wait for all of that crate's dependencies to be completely finished compiling, instead it only needs to wait for each dependency to produce its metadata. The prototype here builds upon that existing machinery by causing metadata to be written earlier than it would otherwise be, prior to complete typechecking. It has the potential to increase the level of effective parallelism, so whether or not this has a benefit for a particular crate graph will depend on whether or not all of your cores are already effectively occupied during compilation or whether Cargo is letting cores lie idle while waiting on metadata to be produced.
Yes, if you mostly have no extra cores already or if you have very wide unrelated deps, this mostly does nothing. Typically, this is not the case though so there is some good perf gains.
Talking with them presently! This specific set of changes won't get in (LLM written, little to no thinking through of the broader design), but I am hoping something like it shows up sometime.
Never thought of it like this, that synthetic code doesn't need to get into a codebase for it to benefit from the discussion around what was encoded with LLM.
(Copying and pasting from what I posted in rust zulip)
There is an .early-rmeta file that is generated that contains the result of type checking the outer part of the functions (arguments, returns) that is then passed down to all the other dependencies that can use it and get a "headstart" on their work. -Zearly-metadata is the flag for Rustc here to turn that on. It also needed to learn how to swap the real metadata in for the early stuff that it was previously using, once the real stuff showed up.
Rustc needs to be able to pause and resume at certain points and Cargo needs to be know how to look for and react to that, which is what -Zheadstart does for Cargo. So there's some changes job_queue that happen to there. I haven't checked this directly yet, but my understanding is that it doesn't kill the process, it just has it sit and wait, which normally would occupy a slot, but doesn't any longer.
---
So not a cache, just being able to pause and start work and better utilize the available slots.
Sounds like if this was incorporated you could include metadata files in crates and even avoid that analysis cost being repeated? Something like typescripts.d.ts files?
It's deeper pipelining of the build units (similar to how CPU does pipelining). Rust has had pipelined builds for a while, but this extends it further.
From what I understood it's more like trust the external API of functions and continue with stuff that is downstream from that in parallel. If the function body then fails to type check then the compilation can be aborted and then you've done extra work, but that's a pretty rare case afaik.
As an aside, it's just silly to make the repository a bunch of checked-in patches. Git already does revision control. You don't need to do revision control in your revision control.
I wanted everything in one repo and this worked well enough. For presenting to rust compiler team I forked the repos in question and made branches for review with the patches.
Seems like an easy win! I'm kind of surprised nobody did this already. I guess someone will need to reimplement this by hand given Rust's AI policy, but this is still great because it demonstrates that it's a really good idea.
Rust has been taking steps in this direction for several years now, having first added pipelining via eager metadata emission in 2019: https://news.ycombinator.com/item?id=49924257 , but there remain some decisions to be made regarding what to do when encountering errors in crates that were speculatively approved.
There’s an interesting comment thread (including potential downsides) about this technique in the submission for “How to speed up the Rust compiler in September 2026” from a couple days ago: https://news.ycombinator.com/item?id=49923594
The commenter there is the OP, for what it's worth
Additional information about potential downsides and prior art (including links to the previous time that this was attempted, in a slightly different form): https://github.com/PowderworksCode/headstart/blob/main/docs/...
Huh, I kinda thought something like this was already being done, but perhaps it was a bit later stage. Neat!
A kind of related shower thought I had, could rustc not instantiate generic methods, but write out what it expects to find (std::Vec<String>::push), and a separate build process listens to these, but makes sure already built ones are not built multiple times from the whole build.
I don’t actually know how much of duplication there usually is, but I had gotten the impression that it would be part of the problem.
Of course I’m no compiler engineer, and I’m sure there are complications like per crate build profiles, etc.
> I kinda thought something like this was already being done
It is. When building a crate, Cargo doesn't need to wait for all of that crate's dependencies to be completely finished compiling, instead it only needs to wait for each dependency to produce its metadata. The prototype here builds upon that existing machinery by causing metadata to be written earlier than it would otherwise be, prior to complete typechecking. It has the potential to increase the level of effective parallelism, so whether or not this has a benefit for a particular crate graph will depend on whether or not all of your cores are already effectively occupied during compilation or whether Cargo is letting cores lie idle while waiting on metadata to be produced.
Yes, if you mostly have no extra cores already or if you have very wide unrelated deps, this mostly does nothing. Typically, this is not the case though so there is some good perf gains.
Excited to see if there is a path to getting this in the mainline compiler
Talking with them presently! This specific set of changes won't get in (LLM written, little to no thinking through of the broader design), but I am hoping something like it shows up sometime.
Never thought of it like this, that synthetic code doesn't need to get into a codebase for it to benefit from the discussion around what was encoded with LLM.
4 replies →
So this is like a cache? Reminded me of https://turborepo.dev for TS, is it a similar concept?
(Copying and pasting from what I posted in rust zulip)
There is an .early-rmeta file that is generated that contains the result of type checking the outer part of the functions (arguments, returns) that is then passed down to all the other dependencies that can use it and get a "headstart" on their work. -Zearly-metadata is the flag for Rustc here to turn that on. It also needed to learn how to swap the real metadata in for the early stuff that it was previously using, once the real stuff showed up.
Rustc needs to be able to pause and resume at certain points and Cargo needs to be know how to look for and react to that, which is what -Zheadstart does for Cargo. So there's some changes job_queue that happen to there. I haven't checked this directly yet, but my understanding is that it doesn't kill the process, it just has it sit and wait, which normally would occupy a slot, but doesn't any longer. ---
So not a cache, just being able to pause and start work and better utilize the available slots.
Sounds like if this was incorporated you could include metadata files in crates and even avoid that analysis cost being repeated? Something like typescripts.d.ts files?
1 reply →
It's deeper pipelining of the build units (similar to how CPU does pipelining). Rust has had pipelined builds for a while, but this extends it further.
From what I understood it's more like trust the external API of functions and continue with stuff that is downstream from that in parallel. If the function body then fails to type check then the compilation can be aborted and then you've done extra work, but that's a pretty rare case afaik.
It almost sounds like optimistic branch prediction when you put it that way
ah ok, so more like an async type checking
Woah nice thought! :) good idea
Those are big improvements!
As an aside, it's just silly to make the repository a bunch of checked-in patches. Git already does revision control. You don't need to do revision control in your revision control.
Disagree. This was incredibly easy to test in a Bazel monorepo due to the ability to inject patch files into 3rd party dependencies (https://bazel.build/rules/lib/globals/module#parameters-13).
I would be very curious to know how that test went please!
I wanted everything in one repo and this worked well enough. For presenting to rust compiler team I forked the repos in question and made branches for review with the patches.
[flagged]
Seems like an easy win! I'm kind of surprised nobody did this already. I guess someone will need to reimplement this by hand given Rust's AI policy, but this is still great because it demonstrates that it's a really good idea.
Rust has been taking steps in this direction for several years now, having first added pipelining via eager metadata emission in 2019: https://news.ycombinator.com/item?id=49924257 , but there remain some decisions to be made regarding what to do when encountering errors in crates that were speculatively approved.
Uhm... emit the errors and fail the compilation? What else would you do?
Maybe there's some work to do to make that happen cleanly but the desired behaviour seems pretty obvious.