Comment by armchairhacker
11 hours ago
A compiler is an algorithm and Claude isn't*. A compiler almost never produces a wrong output, even when compiling an extremely complicated program. But a compiler must be clearly defined and is limited to input/commands it's defined for. A compiler will "correctly" process input into unintuitive output, but that's not always what the user intended (e.g. omitting large sections of code that are undefined behavior). Claude makes mistakes, but can process any input/command expressible in training data, and has more potential to infer when something is unintentional.
Tools like Claude are great at building algorithms, then tests to verify them. The algorithms they build aren't just compilers, but any software that can be verified. Even though Claude may not be a compiler, it has the potential to (at least if it improves) create production-grade software that can be verified, like other compilers. But maybe not good UX without human input (expanded to any subjective experience, like video games).
* Not a "source language to target language" deductive algorithm. Technically Claude is an algorithm to predict the next token, but acting as a compiler or anything else it's an inductive heuristic, because it guesses (https://stackoverflow.com/q/2334225)
> A compiler will "correctly" process input into unintuitive output, but that's not always what the user intended
Importantly, this behavior is deterministic. If the compiler takes your code and spits out something you did not expect, it will always spit out that same result, given the same input. Once you figure out what went wrong, you've solved that problem.
Generative AI is intentionally not deterministic, and that means you're going to solve the same issues -- hallucinations, primarily -- over and over again.
Determinism isn't the main hurdle. It would be a moderate effort to engineer a deterministic claude that would be no more useful than the current one.
The key characteristic of a compiler is that it takes a source language with a well-defined meaning to a target language with a well-defined meaning.
The reason claude fails to be a compiler is that its input language (english) can't be precisely defined. (And even if that were possible, it would very often fail to preserve the meaning when "compiling")
AI could be deterministic, only it requires sacrificing performance and nobody cares enough. Although algorithms are more predictable, and you can follow an algorithm's steps (even a complex algorithm can be broken into large sequences of steps, which may be broken further).
Determinism is the wrong thing to worry about. As you say an LLM could theoretically be deterministic.
Chaos (prompt instability) is the problem that isn’t solvable if you want to treat an LLM like a compiler by doing something like checking in the prompts and rebuilding the code each time.
In general if you change a single line of source code, a compiler will not produce a completely different program. If you change a single word in a prompt, you will get output that likely doesn’t match a single line of the original output.
Chaos is what leads to unpredictability, and there’s no solution to this on the horizon.
3 replies →
“Generative AI is intentionally not deterministic, and that means you're going to solve the same issues -- hallucinations, primarily -- over and over again.”
I see it differently. Sure there are still some hallucinations (but less and less as we progress on), but I see the AI-is-not-deterministic characteristic as a good thing, in the same way that humans and human designers not being deterministic is a good thing. Imagine giving a project/task to a human, and it was completed exactly in the same way with exactly the same outputs as another human that the project was tasked to. Would that be better than each human having varying outcomes of the project? I think most would agree the answer is no.
I agree with what you're putting out there. Every time I see one of these LLM/compiler pieces, the only thing I can think of is that these people don't know what a compiler actually is.
>A compiler almost never produces a wrong output
back log of compiler's bugs can be pretty large
In a production-grade compiler like LLVM they are rare enough that it compiles giant projects like Linux and Chromium without detectable issues.
Maybe eventually Claude can, since it can write tests, but compilers / (deductive) algorithms have other advantages like efficiency and predictability.
I guess we can come back in 20 years and see if Claude is as reliable as LLVM is today. That would give a comparison between similar aged products.
2 replies →
we could make a similar observation about determinism. given a set of inputs, the compiler will create the same output each time.
not always true. but very close (pathological coding defects and explicit randomization excluded)
very different than an LLM.
> almost never
But not never. "Almost" is doing a lot of work there.
> A compiler is an algorithm and Claude isn't
"Algorithm" is not a word with a definition, so I can't say that you're wrong, but I struggle to see how you conceptualize "algorithm" in a way that does include a compiler but does not include a language model.
I think the distinction you're trying to draw is better captured by comparing inductive and deductive program synthesis.
In computer science, one definition of algorithm is basically any program that runs on a turing machine. By that definition, any LLM is an algorithm.
A compiler is a deterministic algorithm. Clause is not.
Of course Claude is an algorithm, but it's a very vague, fuzzy and complex one, where you have no way to determine what's going on inside, and no way to know up front what's going to come out.
Those are the exact concepts I was looking for.
> "Algorithm" is not a word with a definition
what?
Rogers "Theory of Recursive Functions and Effective Computability", page 1, emphasis is the author's:
§ 1.1 The informal notion of algorithm
In this chapter we give a formal (i.e., mathematically exact) characterization of recursive function. The concept is basic for the remainder of the book. It is one way of making precise the informal mathematical notion of function computable "by algorithm" or "by effective procedure". In this section, as a preliminary to the formal characterization, we discuss certain aspects of the informal notions of algorithm and function computable by algorithm as they occur in mathematics.
I don't understand why you people act like you're stumped by literally, literally page 1 of computer science.
5 replies →
If you pair LLMs/AIs like Claude with a formal verification framework you essentially have a new sort of compiler.
That’s technically correct, but it still means that someone has to write the formal specs that Claude’s output is verified against, and someone has to design the formal languages the specs are written in. And once you have formal specs that truly covers all aspects that you ever will care about, it’s unclear if you couldn’t instead build a non-LLM mechanism that efficiently and deterministically spits out an implementation of such a spec.
I expect in the future it will become hybrid. Formal specs provide guardrails for LLMs / AI but within those guards LLMs will author traditional compiler programs and at times drop-down to manually compile / prove certain snippets.
Could we describe the first humans manually writing Assembly and their own higher level abstractions as "compilers"? Loosely I think we could, and LLMs fit the same mold.
1 reply →
[flagged]