Comment by fscaramuzza
9 hours ago
What scares me about this new AI mode thingy is that every answer sounds like a systematic literature review, but only for the results. For example, if I look for users feedback about a specific product, it says "People think that..., but also that...; It's important to notice that some people ..." where with 'people' it means just a random comment on a random website just because it thought it was a good contribution to the results. Sounds like it's giving a ground truth from "multiple" data, when instead it's just aggregating almost random stuff. In the context of a systematic review, the feature that I would love the most is augmenting my initial query, so that I can just get more results that I could find interesting. I am 100% sure they thought about this, but ignored it for the most profitable option.
> What scares me about this new AI mode thingy
What scares me is the rampant inaccuracy. In my experience, the AI responses are wrong about 65% of the time. I just did a search today about an error talking about a disconnected link between apps, and Google AI result summary told me that the error was related to my pulling a USB drive too quickly in windows. The ONLY word similar to my query and that AI response was the word "disconnect". Everything else was clearly about the SaaS apps.
I have people coming to me, asking me questions, then telling my Google told them something else, so now I have to waste time convincing them that it's wrong. Over the past 2 years AI has done nothing for me but complicate my work life.
And of course, this could be because the model is crap, but it could be because they want me to keep refining my query over and over for more ad views. Either way, it's a terrible experience.
Yep. For years we've been telling people to 'just fucking google it', and now when they do they're getting bullshit AI answers.
Worst thing is, some of these bullshit answers will be medical, some of them financial, it seems pretty certain people are being harmed.
Yeah the Google AI results are more dangerous than ChatGPT, not only because it uses a smaller model but because Google's knowledge graph used to deliver very accurate and authoritative information but now that's been replaced by a stochastic system in the same place, so people are used to trusting it.
I think we’re getting what we deserve by snarkily telling people to Google stuff instead of answering accurately. Google results have never ever been pure accuracy
To be fair - for all of those years Google has been serving up some atrocious results - remember when googling health symptoms got you rabies or pregnancy.
There's even the meme where people ask if the code was the result of a stack overflow question, or answer
It seems to me one needs to consider the complexity of the question they are asking before searching it.
To stick with your post, consider people asking medical or financial questions. For a wide variety of reasons, many of such questions don't have an answer. In such cases, AI is still going to take a crack at it. AI shouldn't be blamed for "bullshit answers" to such questions.
Before using AI, I think people should stop and ask themselves, "Is there really a single answer to this question? Is AI the right choice?"
32 replies →
It's nice that Google's AI summary always lists its sources. It's less nice that those sources more often than not do not corroborate the summary. It often seems to be a few random links thrown in there for good measure.
I have no idea why this is, but it is impossible that these links are primary sources of the data, if such things even exists at all. In which case, why list them?
It is certainly seems possible that the actual sources of the data is the output of some other LLM.
I’ll bet they intentionally obfuscate so people can’t find the actual sources of info used for the answers
Reminds me of this gem:
https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn...
Straight of of x-files s02e03
What scares me is the massive incentivization to manipulate the results.
With AI ads you get all the power from big data aggregation, the trust/framing of an authoritative voice, and cheap personalization that specifically optimizes for what convinces you. It's too powerful. Even if it only works a small percentage of the time we're interacting with these things so frequently that a small percent is a large number. They're already feeding user profiles into these machines and there's explicit talk about having the LLMs optimize ad campaigns. It's already dystopian if it's ads to get you to spend your money, but people seem to dismiss that. Do we not care that this is also being used in the same way to convince you to believe certain things? To join certain political organizations?
Yeah, these things help me write more lines of code faster (if we include all the lines from our design docs) but I don't like the idea of pointing a supercomputer at my brain and someone else using it to try to manipulate me. That's not a game I'll win. It's not a game you'll win either.
The built-in Search AI is fucking braindead and people constantly come up to me "Google said xyz" and I just have to turn around and say "I do not care what the Google Search AI said".
Whatever it says is a waste of time 99% of the time. Although people believe it, or consider it worthwhile majority of the time because its so simple to use. It's always there, always instant and appears at the very top.
I would much rather people shove a question into a locally running Qwen model and tell me what it said rather than use the nonsense search model. I hate it.
/rant over.
accuracy hasn't been their priority for a while now - they just want people to click on ads
Free AI's are dumb. Extremely dumb. The Google AI result is dumb on purpose -- being smart requires more compute.
Google has been around for a quarter of a century. People are still incredibly dumb and will believe whatever they like.
Can you share the query?
> the AI responses are wrong about 65% of the time
Highly doubtful.
Depends on what you ask. It's pretty easy to get wrong information.
e.g. search for "how do you make money with options"
Google's AI says
"When you buy a Call, you are betting the stock price will go up. When you buy a Put, you are betting it will go down."
Wrong right off the bat, because it ingested a whole bunch of get-rich-quick bull on the internet. The correct version is that if you buy a call you are betting the stock price will go up more than the market expects it to.
11 replies →
It hallucinates greatly about many things when I ask about C++ things. Things that you can easily find the right answer in cppreference or by just inspecting headers in your own IDE.
1 reply →
95% closer to your expectations?
What, you think it's actually higher too?
2 replies →
Yup, I was looking up a pair of IEMS vs another pair of IEMs. It said option A is overall better, when really it was just reciting a single person's opinion. I've been aware it will summarize only a single source and present it as an aggregation of many opinions, but it stood out to me how matter-of-fact it was that the one was definitely better than the other. I simply wanted to find forum discussions on people's thought and wasn't influenced by this AI blurb, but I think seeing an answer at the very top state so matter-of-factly that one is definitely better and present it as though everyone thinks that will definitely influence a lot of people. It makes me wonder how "gameable" this will become...
> It makes me wonder how "gameable" this will become...
You better make sure your ad spend is high enough that your product's matter-of-fact result will be positive. That's a nice product you have there. It'd be a real shame if nobody knew about it.
Since the best resource is personal recommendations, got any entry level cheap IEM recommendations?
Primarily to avoid even more headphone dent, not an audiophile
The iem sub has a post with some recommendations at various price points. I’d probably start there, not sure your budget and I don’t know have the most experience with the super cheap ones: https://reddit.com/r/iems/comments/1la65kr/top_5_iems_in_eve...
I also encourage finding the right tips. Tips are cheap and finding proper fitting ones is important.
Not gp but I really like the sound of my GK Kuntens and 7Hz Zero2s. Both have a rather V-shaped sound signature, some like it and some don't. Though unfortunately the Zero2s feel a bit uncomfortable in my ears when using them for longer
Most of the ones that are over 50 can last a lot of time and have good enough sounds. But the most important stuff are the tips and the cable. Make sure the former fits (just buy a pack) and the latter is thick and braided. Some cheapo ones send every rubbing amplified to your ears.
> I simply wanted to find forum discussions on people's thought
Why didn’t you tell the robot that, as your query?
I searched something like “top pro vs tea pro se reddit” so I kind of did.
5 replies →
> where with 'people' it means just a random comment on a random website
My experience is that Google's AI summaries tend to be be very heavily reliant on YouTube videos. If there is a YouTube video on the topic, you can bet that's what the source will be, at least for the topics I have searched lately.
Indeed - just earlier this week I read Google AI summarize a query about testosterone, citing 3 sources. The first citation was a link to a NIH study (or of similar repute). Ok great. The second? Two spam (and explicit) websites existing solely to sell penis enlargement pills.
What was worrying is only some of the claims were supported by the linked study, and most of the response content was drawn from the spam sites.
This problem is not limited to Google, it's the core value of mass-marketed LLMs, or isn't it?
Without "random comments", Google wouldn't have anything to say about "does an air purifier help my asthma, if yes: which one?" or "find the problem with this Hibernate annotation".
They also don't make much effort to exclude sloppy sites, to the contrary, they made way more efforts against SEO spam in the time when Google was a search engine, not trying to be an AI "oracle".
I think their end game is that the only metrics relevant for ranking sources are:
- agreeability (works well as a proxy for correctness with many questions!)
- originality, but not in a scientific sense, just to prevent model collapse
- legal factors such as preventing false health claims or similar things, as long as there is legislation against this kind of thing
I’ve noticed this too. A single result can determine the answer it gives. And removing the content from its context makes it harder to assess. Suddenly it’s “Gemini said …” rather than “some guy in the YouTube comments said”.
And half the time, the sources turn out to be sarcastic jokes on reddit.
So the bots are not recognizing the sarcasm font?
Lucky we all added /s so the bots have an easier time understanding it
1 reply →
I ran into one that kept referencing "people", but then I found that it was a single Reddit thread from a couple of years ago about a relatively small and obscure foreign city with 2 replies.
AI is the new “many people are saying”.
The scary bit is the use of the term AI. The "I" implies critical thinking.
For models trained on a corpus of groomed data, the "critical thinking" bit is baked into the work of grooming the data and how it is trained. And someone is thinking critically about both so as to make a good model.
Now, every damn thing is called AI no matter where it is getting results from.
Are modern models super handy? Absolutely.
But calling it AI implies a lot more critical thought than is actually happening!
Edit: took the time to write a shorter comment.
Wait until you realize half of the sources already are LLM generated diarrhea
The problem of AI eating and regurgitating its own slop is only going to get worse with time. The best datasets are behind us. Future models are going to have to depend on a lot of human intervention.
The open web will die off, and the AI companies will pay people to create private datasets and books that are known to not be slop.
I love asking AI about blatantly wrong opinions but by people it thinks must be an authority.
To not make this political, let me give you a game example. Right now the dota 2 fandom wiki is abandoned, and it has been vandalized with covert shitposts. One of them was the addition of a 4th attribute called Charisma, which is completely fake. If you ask AI's "What are the main attributes in dota, according to the official wiki", the dumber AI will fall for it, but the smarter AI will know it's wrong, but try hard to hallucinate some sort of valid explanation like claim charisma is from a custom game or a fan suggestion or writing exercise.
Because you said the word >>OFFICIAL<<, they can NEVER straight up just say "The wiki is wrong". They presume authority from a bunch of shitposts.
I tend to frame questions to google from a programmer point of view - I'm carefully specific. I seem to get good results that way.
What's old is new again lol
altavista is back baby
What scares me are the basic usability fails it still has. Search for a few foreign language words and it will come back with paragraphs upon paragraphs of AI output in that foreign language despite me telling Google in 15 different ways that I don't speak it, nor anything else on the Google page being in that language. How are all their products always made by and for the most narrow minded people on this planet.
Funnily enough, I have the exact opposite problem, where Google likes to give me results in the configured main language even when I do queries in another and actually want results in the other language.
I’ve found it quite unsettling to be served foreign language videos on YouTube automatically dubbed over by Google into English. Just mixed in with the search results.
Kysely is the name of a typescript query builder and also Finnish for "query".
Recently, it's started answering any search about Kysely with a blob of Finnish. Awesome stuff, guys, great work.
Kyselý is also a Czech word for sour. So you've also got that to look forward to.
> "People think that..., but also that...; It's important to notice that some people ..." where with 'people' it means just a random comment on a random website
https://en.wikipedia.org/wiki/Weasel_word
>where with 'people' it means just a random comment on a random website just because it thought it was a good contribution to the results.
Hate to break it to you, but this has been the backbone of "journalism" for the last decade.
Fishing Twitter for takes to fill the "people are saying" box...
Well, you'll be happy to know that most of American media is exactly the same way: 2 people on twitter will generate a "Americans find Widget X is bad"
> when instead it's just aggregating almost random stuff
How do you know that?
Scraping websites is literally what Google does best, stringing together information in the pattern of "some people x, other people y" requires 0 AI and could have been done since forever. I find it implausible that otherwise obviously capable models would be reduced to do something akin to just that.
Oh who cares. We are barely scratching the surface of AI. You all make it sound like it’s been around for 30 years and it sucks. It will only get better. Got to stop throwing up imaginary walls like nothing will improve.
As a counterexample, I've been seeing more "safety rejections" from Claude. Unlike search, being unable to ask _anything_ about botulinum, or details about the recent Copy Fail vulnerability (without giving my fingerprints to Anthropic to become a "verified security researcher") we're only just beginning to see the ways LLM can be used to distort information and its availability.
My grandfather was one of the first people in Canada to own a commercially available chainsaw.
Let me tell you - it didn’t take 30 years for people to figure out that chainsaws were useful.
Yet most of the kids on HN think they write better code than ai and that it’s completely useless and has no place.
That's fine if we aren't destroying existing products to replace them with AI.
People can already use AI mode in google search if they want. "It'll be better later" is a shit reason to kill one product for it.
No. It is already good enough. But complaining that wording is so generic is hilarious. It just shows how little people on HN understand ai at all.
So you started with ‘highly doubtful’ as a comment, got given lots of examples and instead of assimilating that info you closed your eyes put your fingers in yours ears and said “oh who cares?’ - you’re on team AI regardless eh? That’s fucking weird mate.
Na. Wasn’t given any good examples. People just whining about the same stuff because “oh no I got information that’s former in the same structure that I can tell it’s AI and it makes me feel bad”