Comment by OtomotO
1 day ago
What if there was some malicious code within the 1 million previous lines?
The problem is trust... About nobody has the time and the resources to read 1 million lines of code, keep everything in their head and derive whether there are problems...
The other is pure complexity. We can reasonably trust the Zig implementation because it had a lot of talented people working on it. But then they threw it out and rewrote it in Rust, and it's like okay a huge number of tests pass but:
1. Are we positive the AI didn't cheat when getting some or most of them to pass? Are we sure it didn't just cheat on even 1 percent of them?
2. Are we positive that the AI didn't introduce bugs that the tests didn't think about because there was no need to think about that before (because actual people understood the code)? And therefore, those things go untested, and nobody can write good competent tests for them because nobody understands the code in the first place.
[dead]