Comment by kijashdkujdfhas
17 days ago
"AI can make mistakes" continues to be the fundamental problem.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
That's fundamental to how _any_ technology works. If the bar for use is "can never go wrong ever", then you need to throw away every human invention ever made.
The problem is the scope of errors is so wide. It's not reasonable to compare LLM style failures to other technology failures. These 'failures' are not actually failures, they're part of the normal operation of the system.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.