Comment by amelius
17 days ago
AI is irresponsible technology, because we have no idea how it works or how to contain it. It's worse than writing code that suffers from SQL injection vulnerabilities because at least in that case we know how to fix it.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you. This may change if we get more legislation. But given the argument above, it will probably never shift to the AI itself having any responsibility. Instead the AI labs can be responsible.
"AI can make mistakes" continues to be the fundamental problem.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
That's fundamental to how _any_ technology works. If the bar for use is "can never go wrong ever", then you need to throw away every human invention ever made.
The problem is the scope of errors is so wide. It's not reasonable to compare LLM style failures to other technology failures. These 'failures' are not actually failures, they're part of the normal operation of the system.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.