Comment by nolok
9 months ago
The fact that it's so eager to hallucinate random things that sounds plausible enough if you're not paying attention without warning you or giving any error should make people reconsider using it for "data journalism" or similar.
If you make your system and it "works", then how will you see the one time out of X where it confidently provides you false information that you happily use because it usually work ?
> how will you see the one time out of X where it confidently provides you false information that you happily use because it usually work ?
You don’t. You treat it like you would a human worker: set your process to detect or tolerate wrong output. If you can't, don't apply this tool to your work.
This is true but misses a key fact, that typical llm errors are different to human errors. Not that they're worse or better but just that you need to understand where and when they're more likely to make mistakes and how to manage that.
Right, that's why I've been recommending dedicated OCR tools (Textract etc) over vision LLMs: https://simonwillison.net/2024/Apr/17/ai-for-data-journalism...