Comment by galangalalgol
7 hours ago
In my experience, it will go to every length to convince you it has ported code that it hasn't. It will silently drop upstream unit tests it has no code for, if questioned it will insert some markdown giving some rational why that specific bit was deferred, after the fact, and then point you to it. This was Opus 4.6 for reference. I have had much better luck with small numbers of higher level tests that I can individually verify equivalency. I haven't tried having it do differential fuzzing on some high level interface, that seems like it might work in some cases. But to summarize, treat it like an adversary trying to deceive you.
No comments yet
Contribute on Hacker News ↗