Comment by DebtDeflation
1 day ago
Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.
1 day ago
Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.
The other day I asked which trains stop at a particular station and it pointed me at a map that isn't real.
It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.
It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.
It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.
It is better than it was but it is not adequate yet.
To give a specific example of where I've seen this utterly fail... A few months ago one of my pets had an unexpected reaction to a new medication. It was late at night, well past our vet's hours. Google's AI response was basicaly "They are going to die, rush to an emergy clinic now".
But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.
Realize that all the Google thing is doing is having an LLM summarize the search results in the page. If the page results are crap, the LLM content will also be crap. It's the LLM version of the "quick result" boxes that would shove wikipedia in your face.
If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.
But it did not? It said something completely differemt then the resources.
A large portion of the time I google something that I know a lot about the AI summary is misleading if not outright wrong. Frightening when you consider how many take their so-called AI as gospel.
God damn. I respect you for being honest, but… god damn. That’s a pretty low bar.
It's just a google search but you can explain what you're searching for. AI summary may contain misleading details if you read it and take it literally, but 90% of the time it gives me pointer for the thing I need and I don't have to scroll through results that often could be just as misleading if not more.
It's demonstrably not. This summary, somehow, often fails to search, or fails to read what it searches and gives you the opposite answer. You shouldn't trust it.
1 reply →
Just in the last few days I had an example where the Google Search version of Gemini had a much better and more comprehensive answer, by far, than ChatGPT. It had to do with Amazon’s use of Data Matrix barcodes on shopping bags. Gemini was able to fully describe how they’re used in Amazon’s logistics chain. ChatGPT essentially said it didn’t know. Your preconceptions may not be accurate.
People are really slow to update their priors after bad experience with an early version
1 reply →
They might be, but still, I’ll be difficult to convince that Google’s shitty-ass 10B params instant answer models are good at anything other than cookie recipes and producing farts real fast for real cheap.
For what it’s worth, any time I see that feature, it’s all farts.
1 reply →