Comment by keeda
2 hours ago
> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.
I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-)
It does highlight the problem facing any search engine though. AI-generated spam will be much harder to defend against with traditional, statistical mechanisms. And this is before we get to the existential problem of prompt injection.
Maybe this is where news organizations can win back their proper place in their relationship with Big Tech: by becoming the sources of verified, vetted information that LLMs can trust blindly. Possibly that's what deals like the OpenAI / Atlantic one are about.
> How is it different from a person believing whatever they read on the Internet?
The problem is LLMs have no capacity for shame.
My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that.
To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and stupid over getting scammed. There's no giant blinking red light that says, "Never let this happen again!"
> The problem is LLMs have no capacity for shame.
I know what you mean but I can't help but be cheeky: https://www.fastcompany.com/91383271/googles-chatbot-apologi...
Jokes aside, shame does not change the underlying point though. Despite feeling ashamed for being tricked, as you point out people can still get scammed again by different tricks. I think your point is more about learning from mistakes than shame.
Which still does not change the underlying point, I suppose. Offhand I cannot think of anything that would fix this problem for LLMs that wouldn't also fix it for humans, like relying on trusted sources.
>The problem is LLMs have no capacity for shame
You seem to be implying that people do, and I'd like to contest that point gestures wildly at everything
This is a great point. I've added it to my list of things when talking about the limitations of LLM.
IMO we must take it a step further: In this context "the LLM" we're all thinking of can never be taught to feel shame because it doesn't exist, it's a fictional character we humans "see" inside a story being conveyed to us. (The real LLM is an algorithm constantly taking a document and making it slightly longer based on what seems normal for all documents.)
So "the LLM can't feel shame" is true... on the same level that "Dracula thirsts for blood". All we can do is change the story-generator so that the next story has characters that fit better in our imagination.
Perhaps the end state is going to be from the last Hitchhiker's Guide to the Galaxy book, Mostly Harmless:
> Anything that thinks logically can be fooled by something else that thinks at least as logically as it does. The easiest way to fool a completely logical robot is to feed it with the same stimulus sequence over and over again so it gets locked in a loop. This was best demonstrated by the famous Herring Sandwich experiments conducted millennia ago at MISPWOSO (the MaxiMegalon Institute of Slowly and Painfully Working Out the Surprisingly Obvious).
> A robot was programmed to believe that it liked herring sandwiches. This was actually the most difficult part of the whole experiment. Once the robot had been programmed to believe that it liked herring sandwiches, a herring sandwich was placed in front of it. Where upon the robot thought to itself, Ah! A herring sandwich! I like herring sandwiches.
> It would then bend over and scoop up the herring sandwich in its herring sandwich scoop, and then straighten up again. Unfortunately for the robot, it was fashioned in such a way that the action of straightening up caused the herring sandwich to slip straight back off its herring sandwich scoop and fall on to the floor in front of the robot. Whereupon the robot thought to itself, Ah! A herring sandwich...etc., and repeated the same action over and over again. The only thing that prevented the herring sandwich from getting bored with the whole damn business and crawling off in search of other ways of passing the time was that the herring sandwich, being just a bit of dead fish between a couple of slices of bread, was marginally less alert to what was going on than was the robot.
> The scientists at the Institute thus discovered the driving force behind all change, development and innovation in life, which was this: herring sandwiches. They published a paper to this effect, which was widely criticised as being extremely stupid. They checked their figures and realised that what they had actually discovered was “boredom”, or rather, the practical function of boredom. In a fever of excitement they then went on to discover other emotions, Like “irritability”, “depression”, “reluctance”, “ickiness” and so on. The next big breakthrough came when they stopped using herring sandwiches, whereupon a whole welter of new emotions became suddenly available to them for study, such as “relief”, “joy”, “friskiness”, “appetite”, “satisfaction”, and most important of all, the desire for “happiness”. This was the biggest breakthrough of all.
> Vast wodges of complex computer code governing robot behaviour in all possible contingencies could be replaced very simply. All that robots needed was the capacity to be either bored or happy, and a few conditions that needed to be satisfied in order to bring those states about. They would then work the rest out for themselves.
2 replies →
The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information.
Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.
I totally agree, centralization is dangerous, ideally we want any output to be corroborated by multiple, independent sources of truth. But given that the alternative is the absolutely unregulated, unaccountable, wild west of arbitrary content posted on the Internet, I cannot see a solution besides some sort of centralization of trust.
Let say you are a cave dweller and lived your whole life there. I go in and tell you the world is flat and you will believe me. Only way to reject the world is flat would be to go outside of the cave.
ML cannot ever go outside the cave. It does not have real world feedback. It also does not have a will, type of feedback loop, to learn beyond what it was initially trained on.
ML / AI only has the ability to regurgitates what it has been trained on. Garbage in = garbage out. Feeding ML garbage is the real AI wars.
AI will always propitiate misinformation. They even create a marketing term to assist in the sale of lies, hallucination.
https://en.wikipedia.org/wiki/The_Cave_and_the_Light
ML can regurgitation that book and never will be able to apply it.
> How is it different from a person believing whatever they read on the Internet?
Because the answers, while prompting, are clearly more human and charming than a search engine results list?
You and OP are both unnecessarily diminishing what 'glorified search' is.
If you had told me that in 2015, we would have a tool that can iteratively search the world's best and largest unstructured database and synthesize outputs in language (any natural and structured language), I would have said that is basically AGI.
This whole desire for it to 'reason' (autonomously prime its search with a few thousand token) and 'think' (search for the best information within its parameters and synthesize that with its context) is semantic and will feel irrelevant as the technology progresses and we become more used to what these things are actually doing.
I honestly struggle to imagine what AGI will be if not an ever-improving semi-structured database (parametric or otherwise) that we become increasingly good at searching.
If that’s really the case, then I’d say 2015 you needed to do more reading and thinking about AGI and the nature of intelligence and consciousness. The Chinese Room thought experiment is a good starting point for thinking deeper about what AGI is.
But really, I have trouble grasping how anyone can really think database searching is intelligence. For starters, I’d say the capacity to learn on the fly with relatively poor input data is a necessary condition for intelligence, and you can’t get that with database search.
"How is it different from a person believing whatever they read on the Internet?"
Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.
> How is it different from a person believing whatever they read on the Internet?
It's not, directionally. But I think this is kind of bypassing the main point here.
With an LLM's natural tendency to pattern-match in this way, it's easy to see that it can be used to launder disinformation. If in the olden days, I'd done a google search for "worst war criminals" and saw these blue links on that SERP:
"Putin is the 21st century's worst war criminal" - support-ukraine.org
"Zelensky is the real worst war criminal" - publicrelations.government.ru
My takeaway would be that both those are claims made by third parties, one or both could be lying. Even if I only saw more results from one side than the other, most of us understood that the presence in search results doesn't imply Google's endorsement or prove anything besides the fact someone set up a webpage and wrote something.
In contrast, today a lot of people tend to ask ChatGPT something and if it spits back an answer they are - at minimum - being subtly biased that even though it may be in dispute, ChatGPT "agrees" with one position, and that carries at least a little authority. And at worst they wrongly assume that the "correct" answer was selected by deep intelligence, that a lot of data has been analyzed and this answer arrived at, rather than there just being one completely untrusted webpage somewhere that matches their query really well.
And as bad as that is with a "real" model like ChatGPT or Gemini, people also give the same respect to the idiotic, super-fast toy model Google uses for its "AI Overviews"!
>"gullible."
Enough with the anthropomorphization