Comment by wrs
12 hours ago
A few models ago it was not unusual to find that Claude had written tests whose assertions looked correct, but were so thoroughly mocked out that they ran no real code at all — a Potemkin test.
I haven’t seen it do that in quite a while, but it was an interesting failure mode!
I had Sonnet volunteer some tests last year, which really impressed me at the time. It was just a couple console.log statements that printed green checkmarks and "tests passed!"