Comment by ivanzhaowy123
6 hours ago
In my experience, agents often think of more edge cases than humans when writing unit tests. But under the guidance of certain skills, they can become mechanical and lose sight of the business logic.
For example, when I use the Superpowers skill set, the agent proactively adopts TDD for every new feature. But its understanding of testing often stays superficial: if the user asks for a screen with a “Send” button, it first writes a test checking whether the button exists. The test fails, so it adds the button to make it pass.
As a result, the test suite fills up with low-value cases that check whether a property exists or a string matches exactly. The agent follows the “write a failing test, then implement the feature” workflow, but never really tests the business behavior: When should sending be allowed? What should happen after success or failure? How should duplicate submissions be handled?
The problem isn’t that agents can’t write tests. It’s that they seem prone to reducing TDD to a rigid sequence of steps, struggling to independently derive meaningful test cases from business requirements and use them to drive development.
I asked Sonnet to port a prolog interpreter from Python to JS. I was surprised and delighted that it ended up writing a test suite for it. All tests passed.
Until... I inspected the code: it was just a bunch of print statements that said "test passed!" and didn't actually test anything.
That was last year, so hopefully it doesn't do that anymore.
This is not really my experience using TDD with Claude at all. When I tell it “a button should appear under x condition” it writes the test, and will often even verify the test by changing the code to fail the condition correctly before coming back and tell me it’s done.
Every once in a while one of those useless tests you mention will sneak in…but I still read the code and either just remove it or tell the agent to get rid of it. It happens so rarely it’s barely an inconvenience.
But I don’t use any skills or anything like that to drive it. I just have a line in my CLAUDE.md to follow TDD best practices.
Personally I find most skills like these “superpowers” are just bullshit and don’t really help at all.
Probably because thats how most companies do it because most corporate workers are lazy box-tickers who are long fed up with the processes. It really feels like we're forcing human processes onto AI.
What actually is the point of TDD? - If its to force you to think about edge cases early before you've started building the feature then that sounds like a human trait - If its to be living documentation then that sounds like a human trait
We're going into weird rabbit holes where we've mismatched the tool that is AI which produces extremely cheap code very quickly - with the processes that we've built for slow and expensive to write human-generated code.
I think most people miss the point of TDD. It is not just about edge cases. It ought to be:
* A way of matching requirement use cases to tests.
* to modulate the number of tests written not only to make sure you have enough coverage but also to make sure you don't pointlessly cover the same edge cases multiple times.
* a way to cheaply provide feedback and validation on code as you are writing it.
With AI the ability to churn out useless tests has exploded (both with TDD done badly and with no TDD at all) and that has actually incurred a new type of cost we didnt have to face before.