Comment by andai
10 days ago
I have an amusing update to report, I've checked one of the major new features that was added a few days ago. It turns out it was implemented backwards, and in a way which defeats the entire purpose.
Nevertheless, several thousand lines of test code were added, and all tests pass!
I found this particularly amusing because, I've been obsessed with correctness and verification lately, thinking about rewriting the game in Rust, and adding formal verification to the game.
What I realized here is that, this wouldn't have helped at all. The AI would have just written the wrong code in Rust, and then written a mathematical proof that the wrong code is correct...
Agree. The tests are great! Playtesting reveals that gameplay breaks my assumptions on how the system should work. Where to go from here?
Telling AI to "do TDD" is a shortcut that gets me over 80% there by itself in aggregate with little effort. The last 20% is painful. More painful than the usual "last 20%" in game design and development. I think this is part of the human vs. LLM gap.
If you revert the code to before the feature was implemented, then called out the way not to do it (the failed version you're describing) - what would happen? Maybe that "80% there" turns into "90% there", or maybe it's a bit better. Or maybe a bit worse. Working with LLMs vs. humans, does the gap shrink further if you gave other people the same requirements you gave the LLM? I think it depends on who you're working with - other people have their own opinions and don't necessarily just do exactly what they're told without taking their experiences into account.
In a way I see it like playing a game of telephone. I have my understanding of the system, I translate it into words, the LLM interprets it and does what it does based on that. For some reason, it feels like this is a telephone game with 3 people: A to B to C where the translation layer of my thoughts into words is an inserted "B". In contrast, if instead I'm expressing my thoughts to another person it's A to B. This extra connection here is lossy in the same way a game of telephone is lossy from player A to player C.