Comment by lelanthran
14 hours ago
> The code they supplied to me has NEVER worked or was so convoluted that I threw it away and did it myself.
For me, it's like reading prose with "Not X, not Y, just Z": it's technically correct, but grates like fingernails on a chalkboard.
I have real trouble sometimes, reading what SOTA (Fable, etc) generate - no isolation or partitioning at all.
The worst was the planning an AI does. When I plan something, it'll be split according to data structures "An object to hold this, an intermediary for the obejct to talk to ORM, a serialiser for it that does this", etc.
The "plans" from SOTA are sometimes just hilarious. It'll go "phase one, implement these user-facing features. Phase two, implement those user-facing features", etc.
That's not a plan, it's an aspiration! A roadmap maybe. A plan, in my way of working, is a blueprint of where all the data goes, with algorithms connecting them. With AI, the data is incidental, the algorithms are incidental, only the goal (in the form of tests) remain. It'll work out some spur-of-the-moment idea around data at the time of writing.
So yeah, I do what you do and throw their stuff away. Currently having more success laying down a skeleton manually and then asking them to add a single feature at a time.
That's with SOTA models as of September-26-2026.
I think part of the problem -- answering all of why people are somehow OK with this, and even why the AI does this in the first place -- is that the majority of software developers never got to the point of understanding any of this: the lack of understanding how to carefully structure data surrounds a question they don't even know how to pose, much less answer, and so "write some tests then incrementally try to make them work without breaking any of the existing tests" is the only way they know how to develop at all. In the end, that means that, with the current state of the art (which might change, of course... potentially quickly), your strategy of treating the AI as a junior engineer who fundamentally isn't ready to do your senior-level architecture job makes a lot of sense.
Just reading this there appears to be 2 problems.
1) The size of task the AI has been given to do appears to be too big, which is why it looks like a roadmap/aspiration. You can ask it to implement a single feature or even a single part of a feature. Just keep cutting the size of the tasks until you become comfortable with it.
2) You like plans in a particular way following data structures/data etc. have you actually told the models this. It doesn't magically know. For the record the fact that the models focus on the end behaviour covered with tests is the way to to it imo. The actual implementation is less important and can be refactored as you wish fairly easily with the AI with the tests ensuring the feature still works.
I do agree though that the current SOTA models are very keen to just implement absolutely everything straight away without explaining/exploring properly. You can customise it fairly easily by using the various skills/agent/claude files to remember your preferred workflow, imo the agents adhere to theses better than they used to even just a few months ago.
With Claude and Codex, you can specify exactly how you want the plan and code to be researched and written. That goes into your rules file. (Claude.md etc). Also, make them read the existing code so they can follow the patterns.
i'm using LLMs/GenAI to do one thing: write unit tests.
since i really don't follow the idea of "writing unit tests first", i implement the feature, test as a user, and then use LLMLs to write the basic unit test. then, i will write more tests to make sure i'm covering everything.
feels like an ok-ish compromise because LLMs can do some ok job with defensive code, while i maintain the main implementation and more advanced test scenarios.