Comment by silverliver
17 days ago
We are driving full speed into a xerox 2.0 moment and this time we are doing so knowingly. At least with xerox, the errors were out of place and easy to detect by a human. I wonder how many innocent people will lose their lives or be falsely incarcerated because of this. I wonder if we will adapt our systems and procedures to account for hallucinations and "85%" accuracy.
And no, outlawing use the use of AI or increasing liability with its use will have next to nothing to deter its misuse and everyone knows it. My heart goes out to the remaining 15%.
I love generative AI as a technology. But the worst thing about its arrival has been the reckless abandonment of all engineering discipline and common sense. It’s embarrassing.
CCC talk about Xerox copiers changing numbers when doing OCR:
https://media.ccc.de/v/31c3_-_6558_-_de_-_saal_g_-_201412282...
Would be nice to get a translation for a broader audience, glad folks are reporting this out!
There is a translated one: https://www.youtube.com/watch?v=zXXmhxbQ-hk
the first thing that guy says that existing non-AI solutions are not that great. then he says that AI beats them in the accuracy. so i don't quite understand the point you're trying to make here
Humans accept a degree of error for convenience. (driving is one of them). But no, 15% is not the acceptable rate. More like 0.15% to 0.015% depending on the country.
Meh, just maintain an audit log and an escalation subsystem. No need to be luddites when the problems are process, not tech stack.