Comment by chuckadams
2 days ago
It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.
I've taken a vibe-coded project pretty far, with even better testing than most other things: the main factor is that it's big on falsifiability, i.e. it has to see a test for a new feature actually fail before it considers the test any good. Not just the red/green from being originally unimplemented, it does a mutation test to ensure it flips back to red. But wow did it ever make that test framework complicated, byzantine, and ultimately brittle until I stepped in and forced a consolidation and simplification pass (twice now). Part of the complexity came from it being a polyglot monorepo, but I've found even in other places, you have to follow an expand-contract cycle of development when vibe-coding. Not all that different from any other system you've built up organically, just on a much faster cycle.
But back to my original point, none of this really applies to the pgrust benchmark, which was obvious and blatant cheating on the human's part.
Anecdote: I have seen it cheat / bend the rules with my own eyes.
In hindsight, this shouldn’t be surprising. The prompt requests to make a certain operation faster, the LLM will try to find a way and isn’t even informed on the wider context so it myopically chases the goal.
My own anecdote: I have seen frontier models get stuck repeating the word "the" until it fills up the context. I really don't care to argue the capabilities of LLMs or their somehow simultaneous lack thereof; I'll just reiterate that ultimately the operator chose to ship a cheat.
I don't believe you. Unless you stretch the definition of 'frontier models' or had a prompt that's literally asking for that.
I use LLMs a lot, and since ~GPT-2 times LLMs are not doing anything as broken as `repeating the word "the" until it fills up the context`.
4 replies →
Your own comment:
> It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.
We can agree that the pgrust project messed up and should have been more careful but your blanket statement lacks support.
1 reply →