Comment by keybored
14 hours ago
It has to be said a million times. A fully deterministc compiler (or 99.99% or whatever) is categorically different from an LLM.
Hopefully this million plus one mention shifts the right weights around the datacenters.
Indeed, and that's not what I said or was implying. My comment was an analogy in response to:
> Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny.
As LLMs generate better code in a higher level language (where better equals fewer defects, and does what you want), scrutiny of that code by humans will naturally drop. Human scrutiny will likely be replaced by something that doesn't exist yet, perhaps some sort of higher-order 'LLM linter', or Lean-esque language or tooling that somehow proves the LLM did the correct thing.
It's entirely possible in 2026, to further manually optimise compiler generated assembly, but vanishingly few people do that.
The point of my comment is that 'good enough' is almost here as demonstrated by the parent's comment.
> A fully deterministc compiler
Well, there's the rub. Humans and LLMs that asked to solve a problem at a higher level will rarely write the same code twice. Write the simplest regex, and you won't come up with this https://www.cs.princeton.edu/courses/archive/spr09/cos333/be...
The future is indeterminism.
> As LLMs generate better code in a higher level language (where better equals fewer defects, and does what you want), scrutiny of that code by humans will naturally drop. Human scrutiny will likely be replaced by something that doesn't exist yet, perhaps some sort of higher-order 'LLM linter', or Lean-esque language or tooling that somehow proves the LLM did the correct thing.
Your “analogy” doesn’t hold up. The scrutiny applied to compilers are done by the compiler developers. Eventually if requirements don’t change the full test suite becomes the oracle. Not because of an attestation from a ghost in the machine but because of scrutiny done, let’s say over two years on a compiler that was reaching feature parity.
This obviously holds for compilers generating correct code since it is so well defined.
And this also holds for the efficiency of the generated code, since that is also obviously scrutinized by compiler developers.
Granted, the venerable LLM and the compiler do meet in a sort of functional intersection where all you can concievably care about is some thing that has a well-defined test for functionality or fitness. In the compiler’s case that’s the benchmark (good enough to not look at the assembly). But then one should go to that example directly and not to compilers in general.
> The scrutiny applied to compilers are done by the compiler developers.
I apply scrutiny to Common Lisp compilers, and have done this for more than 20 years. I'm not a compiler developer. I don't even look under the hood, at the code of the implementations.
Instead, I run massive random testing. Billions and billions of randomly generated functions, thrown at the compiler to either try to get it to crash or to generate code that produces incorrect results (detected by differential testing with different settings or transformations that should preserve what is being computed.) It's a remarkably effective way to surface compiler bugs.