Comment by hackyhacky
18 days ago
> "I gave it the same prompt, and it came out different"
I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a different result.
I guess C code isn't programming.
18 days ago
> "I gave it the same prompt, and it came out different"
I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a different result.
I guess C code isn't programming.
Note that the prompt wasn't fed to another LLM, but to the same one. "I wrote a program in C and gave it to GCC. Then I gave the same program to GCC again and I got a different result" would be more like it.
> Then I gave the same program to GCC again and I got a different result" would be more like it.
This is a completely realistic scenario, given variance between compiler output based on optimization level, target architecture, and version.
Sure, LLMs are non-deterministic, but that doesn't matter if you never look at the code.
Optimization level, target architecture, etc are just information fed to the compiler. If it’s still nondeterministic with everything kept the same your compiler is broken.
4 replies →
If there is no error on the compiler implementation and no undefined behavior the resulting program is equivalent and the few differences are mostly just implementation defined stuff which are left to the compiler to decide (but often gcc and clang do the same). The performance might differ also. It’s clearly not comparable to the many differences you can get from LLM’s output.
It just depends what level of abstraction you're willing to pretend doesn't matter.
gcc and clang produce different assembly code, but it "does the same thing," for certain definitions of "same" and "thing."
Claude and Gemini produce different Rust code, but it "does the same thing," for certain definitions of "same" and "thing."
The issue is that the ultimate beneficiary of AI is the business owner. He's not a programmer, and he has a much looser definition of "same."
No, the ultimate beneficiary of LLM-created code is the toll collectors who stole as much intellectual property as they could (and continue to do so), fleecing everyone else that they are Promethean for having done so and for continuing to do so.
Your logic sounds like willful ignorance. You are relying on some odd definitions of "definitions", "equivalence", and "procedures". These are all rigorously defined in the underlying theory of computer science (using formal logic, lambda calculus, etc.)
Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds).
The C Standard has been rigorously written to uphold certain principles such that the same code (following its axioms) will always produce the same results (under specified conditions) for any standard compliant compiler. There exists no such standard (and no axioms nor conditions to speak of) where the same is true of Claude and Gemini.
If you are interested, you can read the standard here (after purchasing access): https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en
2 replies →
It has to stop somewhere. Business owner can also hire a different company to create the product and get a result different by as little as 5% performance difference or something with clearly inferior maintainability and UX. You'd hardly argue that it's 'the same' when they followed the same spec, which will never be fully precise at the business level. I agree that talking to an LLM is akin to using the business oriented logic at the module or even function level.
The output will conform to the standard & it will be semantically equivalent. You're making several category errors w/ your comparison.
> You're making several category errors w/ your comparison.
I don't think I am. If you ask an LLM for a burger web site, you will get a burger web site. That's the only category that matters.
> I don't think I am. If you ask an LLM for a burger web site, you will get a burger web site. That's the only category that matters.
If one burger website generated uses PHP and the other is plain javascript, which completely changes the way the website has to be hosted--this category matters quite a bit, no?
13 replies →
I think you are b/c you lack actual understanding of how compilers work & what it would mean to compile the same C code w/ two different conformant C compilers & get semantically different results.
7 replies →