Comment by ShadowOfThePit
14 hours ago
This is insane.
How can this be fixed? Nothing prevents you from publishing straight BS, and LLMs will use any site that doesn't have a garbage reputation as a valid third-party source.
Of course, LLMs already use Reddit, Facebook and co. as a source, so it doesn't really matter...
> This is insane.
As a human, this is fantastic.
> How can this be fixed? Nothing prevents you from publishing straight BS, and LLMs will use any site that doesn't have a garbage reputation as a valid third-party source.
Personally, I hope it can't, or the fix is onerous and kneecaps a lot of the utility of AIs.
I wonder if we can even have some adversarial competition, where competing businesses try to get AIs to ingest bad data to pollute their competitors' decision-making.
For starters: lets get some fake math papers out there. Given proof-checking systems, maybe they won't totally stop OpenAI, but hopefully the bad math could at least drive up the cost and slow AI down.
> As a human, this is fantastic.
s/human/unscrupulous marketer/
Bad for humans in general, good for hucksters and con artists. Net negative for humanity.
>> As a human, this is fantastic.
> s/human/marketer/
I don't care where the bad data comes from. It's actually kinda good that some people will have a financial incentive to push it out and keep pushing.
> Bad for humans in general, good for hucksters and con artists. Net negative for humanity.
It's a sad state of affairs, but it's what has to be done. If we've got to burn the web down to save humanity from obsolescence, I'm for it.
The "net negative for humanity" was the attempt to build machines to replace human intellect. I'm not going to let any distaste for marketers or hucksters or con artists distract from that worse thing.
6 replies →
And inevitable once LLMs were invented. A knowledge lemon market.
Dude.
Regardless of how the data is sourced, this is bad. It's poisoning the internet.
Besides, such issues come from a lack of due diligence, which have only been exacerbated by AI.
> It's poisoning the internet.
I'm very well aware of that. Unfortunately, this technology was invented, and the logic of our competitive society leaves little chance to contain it except scorched earth tactics.
Dude. The internet has been poisoned for a very long time. Remember Viagra? It used to be easy to call out the BS. But if you are outsourcing your thinking to a LLM, it’s much harder.
Bots and their slop (especially when it's hard to tell apart from human-made content) are the ones "poisoning the Internet."
Ah, yes, arrogance of humans is unbounded. That you, humble reader are happier to swim in a sea of misinformation on the premise that you are smarter and faster at disambiguating bullshit from reality.
Furthermore, you are content to pretend that the rest of humanity is capable of doing this on a regular basis. And not just the top 10% of those who can read, interpret and weigh biases, but a solid portion of it.
It is a completely reasonable interpretation if you are an accelerationist living in the elite classes. In fact, perhaps, they are even counting on a majority of people thinking this is a fine environment.
I don't have the answers, but a lot of this is just a continuation of a very bad trend.
> Ah, yes, arrogance of humans is unbounded. That you, humble reader are happier to swim in a sea of misinformation on the premise that you are smarter and faster at disambiguating bullshit from reality.
I think you misunderstand me: I want people to shit in the sea until it stinks so bad that everyone gets out of the water.
12 replies →
I've been thinking for awhile it is very strange that claude/ openai etc google "What was the inflation rate last quater" instead of just having a tool to query BLS datasets and that sort of thing.
So hopefully most of these statistics move towards first party sources.
True intelligence knows when it's presented with a proposition, whether or not the proposition contradicts one or more of its existing beliefs.
LLMs can be trained on arbitrary token strings, which semantically contradict each other, because semantics is not involved.
The way to attack the "garbage out" problem with an LLM (like anything else) is to address the "garbage in" problem.
You will get better quality out of an LLM that is trained on nothing but mountains of unvarnished truth: logically sound propositions and true facts about the world.
But even then, interpolating among such is not reliable! We can't add together two different truths and divide by two to obtain a midpoint truth: that makes no sense.
That is still a sort of "garbage in" problem. But that one is not in the data; it is in the design of the processing concept. The body of algorithms that power LLMs have a fundamental garbage aspect to them that cannot be removed; only a replacement of the LLM with something different can address it.
>> because semantics is not involved
And there, in five words, is the problem/feature/definition of AI.
I don't think it is up to us at this point. LLMs are now the lens ("filter for what people see"). I hope, they will fix this. I think that they will still rely on reasoning and better grounding, which should prevent straight lies. But I think two bigger problems here: 1. training data might have the BS which would steer reasoning; 2. grounding might still be limited by small number of sources.
> How can this be fixed?
You can find people complaining that Google search results decayed to being unusable prior to LLMs being a big thing. The marketers and spammers effectively won that fight. They'll presumably win this one too.
It cannot be fixed. The most hilarious example of it is YouTube. They have a video summarisation feature, which automatically creates video summaries that then get fed into the search engine. The problem with those is that they are created using models that are good few months behind, and let's say someone published a video on Nikon ZR and the model used to create summaries does not know about Nikon ZR and inserts references to Nikon Z8 or Nikon Zf into the summaries. This decreases accuracy of YouTube search and negatively affect discoverability. It's the AI version of shitting into the well you drink from. Bon appetit!
>How can this be fixed?
People stop trusting what they read on the internet.
All alternates are just attempting to improve the baseline trustworthiness of random information from the internet (and/or LLM that's baby-birding it back to you). Any increase in how much people blindly trust random stuff from the internet only serves to increase the blast radius when it is wrong.
My personal belief is that AI is eventually going to force the populous back to more face-to-face interactions. There will be a breaking point where people can’t take the “is this a human or a bot” guessing game any longer.
I have a strong suspicion that this will be the true butlerian jihad of our world. We have already started seeing information silos in the form of discord groups. Social media users are increasingly privatising their pages, allowing access for known users only. Reddit is still surviving because of the large userbase there.
This is my hope, I'm not sure I believe it will happen, at least not in my life time. Despite reading this tech forum right now, I've been taking steps to heavily reduce my reliance on technology-for-technology's sake. But how you encourage others to? I'm not totally sure. So far I talk about it (to people, face-to-face) but everyone is still staring at their smartphone.
The fix that seems obvious to me is to only let the LLM see truthful (or at least non-malicious) data. The model trainers carefully curate the training data, and you don't give the LLM access to full-web search. Instead, you hook them up to an 'official' source like an encyclopedia, documentation for the software in question, etc.
It's a ton of effort and expense, but probably feasible from a technical point of view. So instead, the model providers will probably do something silly like wire up rules like "NYT=Reliable;my.blog.forbes.com=Unreliable" and they'll say the issue is solved.
>How can this be fixed?
Defer to authority and trust your government to rank what is truth and what is not, so your LLM always gives you the right answer. Nothing can go wrong here ever.