Comment by kikdij
4 hours ago
I don't think reproducibility is relevant, here. Otherwise, even your compiler example doesn't stand. No, machine code is not always the same after compiling high-level language. First, it depends on compiler. Clang and GCC won't produce the same output. Then it depends on the version of your compiler. GCC-9 and GCC-11 won't produce the same output. It obviously depends on the architecture too (arm vs x86), but even then, it depends on details about current configuration of the system you're building it on (CPU flags, hardening, etc). And then of course, there may be things randomized on purpose at each build, like randomized memory addresses to make the binary less exploitable.
Yeah I should have been more precise. What I was suggesting is that the behavior of the machine code will be always the same after compiling (assuming no compiler bugs).
Anyway, my work process is as follows: I have the agent write the code, and describe it's purpose etc. via comments so another session can later pick it up easily without me having to explain everything again. That's what I commit. Sometimes I even look at the code :)
What else could I commit? At the current state of the art, I could commit the whole natural language session that lead to the code, instead of the code. When I later check this out in a fresh environment (without the artifacts the LLM putted out), I'd just load the most recent version of the conversation as an initial prompt. Or I could want to check out the conversation at a point two months ago and rebuild the program from there.
Would that produce a program with exactly the same behavior? I don't know. But if it doesn't, the essay's argument is not moot. Not yet ...