Comment by giancarlostoro
6 days ago
> How do people ensure that AI don't produce subtle and stupid mistakes that humans usually don't make, like the one in Amazon that deleted the entire production deployment?
Right... Because Humans have never ever accidentally rm -rf'd a production system in the wrong spot?
GitLab has entered the chat.
Or that time an S3 developer purged more than intended, causing internet outages.
All the above are from 2017.
Then there was a deployment goof by Knight Capital Group, they lost 440 million in 45 minutes. The company went poof back in 2012 as a result.
MySpace back in 2019 deleted a ton of prod data from users during a botched migration (oops).
Humans make mistakes. Anyone arrogant enough to think they only write perfect code is delusional.
We are holding AI to higher standards than we hold humans, who are just as fallible if not worse.
QA your software, stop letting developers test changes.
I was not saying that humans are always right. I was assuming that humans probabilistically would make fewer obvious mistakes, which of course could be wrong and hence my questions. Speaking of QA, we would require AI to generate test cases, right? If so, do we rely on human review to ensure the quality of the test cases?
Correct, and this is why I built out GuardRails (inspired by Beads) because AI just keeps going without checking things. I force my agents to run tests, test if the project builds, and even look for human validation that new features work, as well as ensure tests pass.