Comment by Kerrick
18 hours ago
I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.
A lot of that sense was probably based on side channels like proper grammar, writing style, etc. That’s all gone now :(
No, the sense had nothing to do with the content and everything to do with the context. Perfect grammar and writing style were never enough to get me to trust an anonymous forum post or unknown bloggers post for certain topics like health advice. Sloppy grammar and writing style were never a deterrent for me believing them for other kinds of topics like where to check on the HVAC system to find the sticker. I think the line could more accurately be described as the level of risk if it's wrong.
I dunno, I've read a lot of very well presented nonsense and some very useful insights that were barely readable. I think it's probably useful that people are being trained out of this bias (though of course that's in large part because LLMs do tend to exploit this bias).
For what reason would grammar influence whether something is true or not...
You could tell from tone and polish how much effort someone had put into writing an answer. That was a pretty good signal for some topics on forum sites like Stack Overflow. There were always nuts and cranks who would happily spend an hour writing well-formed prose about nonsense or something obviously wrong, but the eloquent ones were few and far between. Now every crank is equally eloquent and can spit out 1,500 words of passable prose in seconds.
We have other signals now.
Before we would find an intriguing post on the internet from years ago, and you have to verify it with additional research--it's easy to skip that additional research.
With a LLM when you're skeptical you can interogate it. One thing we know for sure is LLMs are quick to admit mistakes were made when interrogated, comically so. A LLM might not always recognize its own mistake, but at least it is available for easy interogation, unlike the forum posts of old.
Manual research from reputable sources remains an option.
You can't interrogate the context of an LLM when you only have its output.
One of the things I do semi-frequently is look for the evidence that some concert took place 15+ years ago. Or maybe I already definitively know it happened, but not exactly at which venue or the exact date of the concert. This I feel like is a non-trivial task, but one with a very definitive answer whose evidence more often than not still exists somewhere online.
In my experience every LLM out there is utterly useless and quickly defaults into "here are other concerts that took place around that time near that location". Google Search (ignoring the AI overview) is even more useless, as it refuses to show literally any webpage that's older than say 5 years. YouTube search is genuinely better than Google at surfacing old and grainy fan-made videos uploaded in like 2010, but also defaults into synonyms nonsense pretty quickly.
But, the search functionality of exactly one forum and three local news websites that I know have an archive that dates back long enough beats every single one of those abovementioned every single time. Three people are talking about their experience at a concert on a random 15+ year old forum thread? It happened. The tiny list of 5 or so (Google-hosted!) Blogspot blogs I have bookmarked? They usually have a photo of the ticket that Google Images refuses to show me.
Not only are search engines completely dead as a category, but LLMs are a shit replacement for them. "We" (okay, Google specifically) has truly committed a crime comparable to burning the Library of Alexandria. Everything older than a decade that wasn't properly documented on Wikipedia is just gone, never to be seen again.
It's vector search that's eaten everything that used to have at least a smidgen of parametric search.
> YouTube search is genuinely better than Google
The funny part of this is that Google search is intentionally bad at returning YouTube videos, presumably because some anti-trust action scared them into artificially ranking videos from local news sites, Facebook, and other ad-walled content ahead of YouTube videos. Seriously, go watch a YouTube video, then try googling its title with “video” appended to it, and see if the “Videos” tab of google search ranks it as the first result.
The most absurd thing that has happened to me more than once is that I found a YouTube video, not by searching through Google, not by searching through YouTube, not by asking an LLM to find it for me, but by finding an old article that embedded it. That embed is of course long broken by the changes on YouTube's side, but once I use inspect element to find its Youtube ID, surprise, surprise, it's still there!
It's usually uploaded by a channel with like 20 subscribers and has maybe like 300 views, but YouTube would rather show me some artist playing a similar genre on the other side of the continent with millions of views that was recently uploaded than a video from an event I specifically typed into a search bar.
Yeah, it's like everyone was under the impression you could just trust the internet before LLMs.
It's a great tool, but verify the important things (or do them yourself)
Notably, it used to take effort to produce crap on the internet, now it’s nearly the default action.
Signal to noise has taken a dramatic hit.
I'm not sure if this is your take, but it feels an aweful lot like an AI "Good enough is good enough" handwave.
For some things, good enough is good enough. For many things, it is not (and neither are random posts on the internet). Before Web 2.0, it was similar to whether I'd trust some random person on the street with it versus going and looking it up in Encyclopaedia Britannica, or the American Heritage Dictionary, or Roget's Thesaurus, or the UC Davis Book of Dogs, or the Cornell Book of Cats, or the Merck Manual, or even the World Almanac or Bartlett’s Quotations if I was feeling petty.
An anonymous answer to a question is more trustworthy to me. They have no reason to lie. They are usually answering out of kindness. At least they used to be. Now it’s often actually bots advertising a product or pushing something, pretending to be a helpful user with an anecdote and a good experience using a niche product.
AI shouldn’t have any reason to lie. But its lies aren’t intentional. It’s just actually making things up and “hallucinating” when it pretends that an option or setting exists, or confidently claims something entirely untrue, and makes up a source to go with it. For something Google is willing to shove into the top of every search result it’s crazy the percentage of time the answer is blatantly incorrect.