Comment by lelanthran
11 hours ago
> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :)
A better question to ask for each snippet is "Estimate the seniority and competence of the developer who wrote the following code, ignoring bugs that linters or LLMs can catch and focus only on structure, maintainability, logical layout and readability."
It almost always estimates the author of my code as above the author of it's own code.
You're asking basically to ignore bugs and correctness. Can it be a useful comparison?
> You're asking basically to ignore bugs and correctness.
Not ignore correctness, just bugs that will be caught by tooling.
> Can it be a useful comparison?
IME, yes. LLMs in an agent-loop are trivially able to write spaghetti code that will never do an off-by-one error or something else that is easily caught by tooling, which is not something humans can do.
Judging code on whether it has bugs easily caught by tooling is pointless - LLMs are running the tooling in a loop anyway, so no matter how bad or poor their code actually is, it never exhibits bugs that are caught by tooling.
> just bugs that will be caught by tooling
Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs".
And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".