Comment by jwr
21 days ago
I'd say this isn't just an AI overview thing. It's a Google thing. Google will sometimes show inaccurate information and there is usually no way to correct it. Various "feedback" forms are mostly ignored.
I had to fight a similar battle with Google Maps, which most people believe to be a source of truth, and it took years until incorrect information was changed. I'm not even sure if it was because of all the feedback I provided.
I see Google as a firehose of information that they spit at me ("feed"), they are too big to be concerned about any inconsistencies, as these don't hurt their business model.
No, this is very much an AI overview thing. In the beginning Google put the most likely-to-match-your-query result at the top, and you could click the link to see whether it answered your question.
Now, frequently, the AI summaries are on top. The AI summary LLM is clearly a very fast, very dumb LLM that’s cheap enough to run on webpage text for every search result.
That was a product decision, and a very bad one. Currently a search for "Suicide Squad" yields
> The phrase "suide side squad" appears to be a misspelling of "Suicide Squad"
Right, the classic google search results are still there. But even before the AI Overview, Google's 'en' plan has been to put as many internal links at the top of the page as possible. I just tried this and you have to scroll way down below the fold to find Barry's homepage or substack.
No, the search queries are likely run through a similar "prompt modification" process as on many AI platforms, and the results themselves aren't ranked anything like they used to be. And, of course, Google killed the functionality of certain operators (+, "", etc.) years ago. Classic Google Search is very much dead.
3 replies →
> That was a product decision, and a very bad one.
I don't know that it's a bad decision, time will judge it. Also, we can expect the quality of the results to improve over time. I think Google saw a real threat to their search business and had to respond.
They are doing an OK job of making AI look like annoying garbage. If that’s the plan… actually, it might be brilliant.
1 reply →
Their "AI Overview" has not noticeably improved on its (many) failings for at least a year. In that time, Google's LLMs have gotten much better. They aren't implementing the advances they've made, presumably for cost reasons.
Meanwhile, every single person I know has come to trust Google less. That will catch up with them eventually.
The threat to their search business had nothing to do with AI but with the insane amount of SEO-ing they allowed to rake in cash. Their results have been garbage for years, even for tech stuff where they traditionally excelled - searching for "what does class X do in .NET" yields several results for paid programming courses rather than the actual answer, and that's not an AI problem.
11 replies →
Back in 2015 I walked 2 miles to a bowling alley tagged on Google maps (in Northwich, England) with my then gf...imagine our surprise when we walked in to a steamy front room and reception desk, my gf asks 'is this the bowling alley' to which a glistening man in a tank top replies 'this is a gay and lesbian sauna love'. We beat a hasty retreat but I imagine they were having more fun than bowling in there
> I'd say this isn't just an AI overview thing. It's a Google thing. Google will sometimes show inaccurate information and there is usually no way to correct it.
Well, in this case the inaccurate information is shown because the AI overview is combining information about two different people, rather than the sources being wrong. With traditional search, any webpages would be talking about one of the two people and contain only information about them. Thus, I'd say that this problem is specific to the AI overview.
The science fiction author Greg Egan has been "battling" with Google for many years because, even though there are zero photos of him on the internet, Google insists that certain photos are of him. This was all well before Google started using AI. He's written about it here: https://gregegan.net/ESSAYS/GOOGLE/Google.html
Google maps is so bad with its auto content. Ultra private country club? Lets mark the cartpaths as full bike paths. Cemetery? Also bike paths. Random spit of sidewalk and grass between an office building and its parking lot? Believe it or not also bike paths.
I went to a party today at a park. Google maps wanted me to drive my car on the walking path to the picnic pavilion. Here, you can get the same directions: https://www.google.com/maps/dir/38.8615917,-77.1034763/Alcov...
Waze (also owned by Google) seems to get it close(r), but it should be noted that actually driving to/from those addresses can't really be done. You can drive to where you might be able to SEE the destination, but not really get there.
https://www.waze.com/live-map/directions/us/va/arlington/alc...
In its defense, it has improved greatly. Back in the day, Google Maps told me to switch ferries in the middle of the sea. The car loading ramp was at an angle, so if I could just build up enough speed...
This really made me laugh. Has Will Ferrell already made a skit for Funny or Die where he precisely follows Google Maps driving instructions and runs over a bunch of old people and children? It could be very funny.
For up to date bike paths, at least where I live I hear very good things about maps.me (based on OSM data).
I mean, that last one sounds functionally useful, since it would indeed be better to take the random concrete paths inside an office property (that wasn’t a closed campus) than to ride on the expressway that fronts it, if the “paths” are going where you’re going.
Yeah it doesn’t really play out like that. Just saw another example today of this gated condo complex where half the sidewalks are arbitrarily full blown bike trails. Clearly they are just trying to automagically get the trails from imagery of putative paths instead of, you know, pulling directly from the municipal bike path network maps. I guess scaling something more like that out was too hard for multibillion dollar google.
I have tried reporting these fake paths in the past but it didn’t get them removed.
Biking is great tho
> It's a Google thing. Google will sometimes show inaccurate information and there is usually no way to correct it.
Surely there is a way to correct it: getting the issue on the front page of HN.
I remember when the biggest gripe I had with Google was that when I searched for Java documentation (by class name), it defaulted to showing me the version for 1.4 instead of 6.
Same problem with LLMs particularly if a new version released in the last year.
Google doesn't really have an incentive to prioritize accuracy at the individual level, especially when the volume of content makes it easy for them to hide behind scale
Well it was accurate if you were asking about the Dave Barry in Dorchester.
He won a Pulitzer too? Small world.