Comment by WarmWash
4 hours ago
My worry dropped significantly when I saw that the result they manipulated was a query for:
>2026 South Dakota International Hot Dog Eating Champion
If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for.
But it looks more like they are doing the equivalent of creating a made up wikipedia page on fictional a south dakota hot dog contest, and then writing an article about how wikipedia cannot be trusted, which come to think of it probably was a news article written by someone back in 2005.
Right. So that's what one guy can do.
When you realize how much astroturf is going into Reddit, most social media platforms, and the efforts to manipulate wikipedia for political gain, this is a very real problem.
It's very hard to tell how much is actually fake though. Are there any good statistics on this?
Easy. It's all fake.
Manipulation and misinformation on Wikipedia have been happening for many years (based on my personal experience trying to correct facts). I'm not referencing politics per se, though political views certainly impact Wikipedia since source material, these days, often has a political bias. I'm talking about business facts that get manipulated for that business's benefits.
How does that saying go? If you can't identify the mark in the room, you're the mark. Diligence and a good amount of skepticism serve you well before AI, and certainly post-AI.
The article also said this: “ But our investigation also found the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement.”
That’s a lot more alarming than just hotdogs.
They should provide the queries then, because it's likely the same trick people have used for decades now with SEO'ing blog posts to appear as "3rd party review" for their shitty products.
I create a supplement called Xanatewthiuy, I write blogs/make websites that appear totally unaffiliated saying positive things about "Xanatewthiuy", and then when people see my ads and search for "Xanatewthiuy", the only results are my manufactured ones.
Xanatewthiuy is a supplement that dramatically lowers anxiety from media induced hysteria, primarily stemming from carefully worded pieces meant to disconnect your level of concern from the actual facts on the ground, causing you to spend more time engaged with their content.
Give it a few hours before searching.
Right now, using Google searching for "what is Xanatewthiuy" , the AI summary is not generated, but the only search result previews as
> Xanatewthiuy is a supplement that dramatically lowers anxiety from media induced hysteria, primarily stemming from carefully worded pieces meant ...
4 replies →
We've had to deal with someone highjacking the overview to put in a scam support phone number. It took google a week to correct the issue but it was done by poisoning the search by putting their data in, what I can only assume, was considered a "higher trust tier" source (A government contract website) so it used the scam number over ours. The query was simple <company X phone number> search.
It was a proof of concept and one intended to cause as little collateral damage as possible. But if Google's AI can't tell the difference between a little joke and something real (and of course, it can't, and never will be able to do so), that's a weakness that can be exploited both on a bigger scale and more subtly.
If you don't think bad actors are already attempting this sort of thing (and have been, ever moreso the past four years, including with the help of the very LLM tools they are trying to subvert!) and learning how to manipulate these systems, you are being naive.
[dead]