Comment by comeonbro
1 day ago
I would propose another mechanism: even the free-tier models have already completely saturated what most people are capable of appreciating.
1 day ago
I would propose another mechanism: even the free-tier models have already completely saturated what most people are capable of appreciating.
Google's free AI responses are moving the bar on what I expect from search. But maybe not in the direction you think.
It's so bad. 3 out of 4 answers for me are wrong.
And I would propose another: the supply hasn't matched the demand. We are still compute hungry and cannot afford the 90% to let their agents run 24/7.
Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.
The other day I asked which trains stop at a particular station and it pointed me at a map that isn't real.
It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.
It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.
It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.
It is better than it was but it is not adequate yet.
To give a specific example of where I've seen this utterly fail... A few months ago one of my pets had an unexpected reaction to a new medication. It was late at night, well past our vet's hours. Google's AI response was basicaly "They are going to die, rush to an emergy clinic now".
But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.
Realize that all the Google thing is doing is having an LLM summarize the search results in the page. If the page results are crap, the LLM content will also be crap. It's the LLM version of the "quick result" boxes that would shove wikipedia in your face.
If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.
2 replies →
A large portion of the time I google something that I know a lot about the AI summary is misleading if not outright wrong. Frightening when you consider how many take their so-called AI as gospel.
God damn. I respect you for being honest, but… god damn. That’s a pretty low bar.
It's just a google search but you can explain what you're searching for. AI summary may contain misleading details if you read it and take it literally, but 90% of the time it gives me pointer for the thing I need and I don't have to scroll through results that often could be just as misleading if not more.
2 replies →
Just in the last few days I had an example where the Google Search version of Gemini had a much better and more comprehensive answer, by far, than ChatGPT. It had to do with Amazon’s use of Data Matrix barcodes on shopping bags. Gemini was able to fully describe how they’re used in Amazon’s logistics chain. ChatGPT essentially said it didn’t know. Your preconceptions may not be accurate.
4 replies →
Or maybe needing.