Comment by johnfn

Your response is definitely on the top 5% of reasonableness from AI skeptics, so I appreciate that :-)

But, if you don't mind me going on a rant: the hallucinations thing. It kind of drives me nuts, because every day someone trots out hallucinations as some epic dunk that proves that AI will never be used in the real world or whatever. I totally hear you and think you are being a lot more reasonable than most (and thank you for that) -- you are saying that AI can get detail-oriented and fiddly math stuff wrong. But as I, my co-workers, and anyone who seriously uses AI in the industry all know, hallucinations are utterly irrelevant to our day-to-day.

My point is that hallucinations are irrelevant because if you use AI seriously for a while you quickly learn what it hallucinates on and what it does not, you build your mental model, and then you spend all your time on the stuff it doesn't hallucinate on, and it adds a fantastic amount of value there, and you are happy, and you ignore the things it is bad at, because why would you use a tool on things it is bad at? Hearing people talk about hallucinations in 2026 sounds to me like someone saying "a hammer will never succeed - I used it to smack a few screws and it NEVER worked!" And then someone added Hammer-doesnt-work-itis to Wikipedia and it got a few citations in Arxiv now it's all people can say when they talk about hammers online, omfg.

So when you say that I should spend more time asking "what do they see that I don't" - I feel quite confident I already know exactly what you see? You see that AI doesn't work in some domains. I quite agree with you that AI doesn't work in some domains. Why is this a surprise? Until 2023 it worked in no domains at all! There is no tool out there that works in every single domain.

But when you see something new, the much more natural question than "what doesn't this work on" is "what does this work on". Because it does work in a lot of domains, and fabulously well at that. Continuously bringing up how it doesn't work in some domain, when everyone is talking about the domains it does work, is just a non-sequitur, like if someone were to hop into a conversation about Rust and talk about how it can't solve your taxes, or a conversation about CSS to say that it isn't turing complete.