Comment by pianopatrick
1 hour ago
At what point should I trust projects like this?
I've heard stories of LLMs changing tests to get all the tests to pass instead of actually fixing code. So I'm not sure I trust it just because tests pass. I also think that tests do not and cannot check everything.
I also don't trust it just because it compiles in Rust. The Rust compiler does not check everything.
So at what point do projects like this cross from "untrusted" to "trusted"?
> I've heard stories of LLMs changing tests to get all the tests to pass instead of actually fixing code.
That's very easy to prevent. Don't let them edit the tests! Run the test suite against a reserved copy.
That makes sense to me as a tactic to prevent that problem.
But the article talks about how so much code was written so fast. Seems to me that to create that much code that fast you have to have AI produce both the code and the tests.
So I am not sure in this project if the AI can edit the tests or not. I assume that because this is Anthropic the AI is doing as much as possible, which would include editing tests.