Comment by fpoling
14 hours ago
The article title is misleading. It is not that Rust compiler was not able to optimize some low-level operations. Rather the author came up with encoding schema that fit most things the interpreter dealt with into 64 bit. This replaced the previous schema that used 128 bit for everything but that can be directly mapped into Rust enums. The catch was that it was necessary to allocate some things on the heap and use pointer indirection but that was used for rare values so on average the new schema provided nice win.
One cannot expect a compiler to come up with such encoding.
> One cannot expect a compiler to come up with such encoding.
One could, however, imagine a sufficiently expressive language that allows the developer to specify the encoding schema without resorting to raw 64-bit words.
This breaks the language. Do you want safety or do you want expression?
What is misleading about the title? A custom encoding scheme is exactly what it suggests. Maybe it has been edited since your comment was posted?
It wasn’t replacing one rust enum, it was replacing what are effectively multiple enums
How so? It's replacing multiple enum variants, but just one enum, "enum Value".
(also; if anything, the title is implying the exact opposite of "Rust compiler was able to optimize ...", "Replacing a Rust [...] with [...]" is clearly moving away from Rust-magic to something else)
4 replies →
I'd actually point at the other half of the title than the existing comment when being pedantic "Replacing [...] with a 64-Bit Word" isn't quite right, it was replaced with a manually packed 64-Bit Word and the occasional heap allocation.
I'm not sure being this pedantic is particularly useful in titles though...
It's worth calling out either way. It's not just an encoding scheme, it's a reasonably significant change in architecture.
When I think of how a "compiler" could make these optimizations, I think the right place is an optimizing LLM (so, just a regular coding agent that you prompted to find optimizations like this one), making the changes in source at the request of the developer. That provides the dev with adequate input on whether they would like to opt-in to an unsafe optimization like this one. The compiler can continue to do deterministically-safe optimizations.
What I'd like to enable this use of LLMs more recklessly is a compiler with formal methods that lets me guarantee equivalence between the opaque optimized code and something actually understandable.
Equivalence on what metrics? In theory what you are asking for makes sense, but I think it is very hard to actually specify what equivalent means in the context of an optimization process that needs to emit code with observably different behavior. Sometimes (although admittedly rarely) speeding up sections of code can even be undesirable for example branchless code for constant time algorithms that avoid timing or energy side channel leaks, or the much more mundane elimination of signed overflow checks or other undefined behavior quirks.
That seems like a horrible idea:
1. Do you really want the rust compiler to run at the speed of an llm?
2. Compiler optimisations are already extremely unpredictable with deterministic compilers[1], I hate to think how unpredictable your compiler would be.
3. What if someone else wants to build the software, do they have to decide on optimisations now? What if the optimisation depends on your features not available on old generations of CPU? (There is a reason we don’t compile with -march=native)
4. Compilers already have “unsafe” optimisations, but people rarely enable them (-ffast-math)
[1]: https://faultlore.com/blah/oops-that-was-important/
> Do you really want the rust compiler to run at the speed of an llm?
That... might actually be an improvement?
1 reply →
You misunderstand me. I'm saying that the developers can make these optimizations with LLMs, at the source level, and thus they don't need to be added to compilers.
Like just open Claude Code and ask it to find optimizations. That's the right place for this kind of optimization.
We'll get there eventually whether we like it or not. Just as soon as the current crop of AI/ML engineers retire, or .. get replaced with bots.