Comment by fkozlowski

So actually I've been testing this — it seems to be useful in limited cases. LLMs have gotten so good now that a properly defined spec is no worse than a spec + property based testing.

I still found it useful for

1. trusting a bit of code is correct

2. rule-dense bits of code (eg something like a date parser)

The real issue so far is that I'm having trouble finding tasks where an LLM can't correctly code up an exhaustively defined spec.