Comment by NBJack
5 days ago
That's yet another band aid on a hemorrhaging patient. And it still assumes the agent(s) are capable of correct comprehension of the purpose.
5 days ago
That's yet another band aid on a hemorrhaging patient. And it still assumes the agent(s) are capable of correct comprehension of the purpose.
Yeah for sure it's a bandaid fix, but it does produce tests that fail if the interface doesn't match the implementation, more often than not doing this does (in my experience).
With a skill like this, which I have written, the LLM will end a session with "wrote 24 tests, 5 failing due to API contract mismatch" or something, and leave it at that. I can then decide if that's a bad interface or bad implementation.