Comment by wkat4242
6 days ago
The problem is the same prompt will yield good results one time and bad results another. The "get better at prompting" is often just an excuse for AI hallucination. Better prompting can help but often it's totally fine, the tech is just not there yet.
While this is true, I have seen this happen enough times to confidently bet all my money that OP will not return and post a link to their incorrect ChatGPT response.
Seemingly basic asks that LLMs consistently get wrong have lots of value to people because they serve as good knowledge/functionality tests.
I don't have to post my chat, someone else already posted a chat claiming ChatGPT gave them correct answers when the answers ChatGPT gave them were all kinds of wrong.
See: https://news.ycombinator.com/item?id=44171575
You can send me all your money via paypal, money order or check.
I'm not gonna go all out, this thread is gonna be dead soon but here all the toothpastes ChatGPT was referring to
[1]https://dentalhealth.com/products/fluoridex-sensitivity-reli...
[2]https://www.fireflysupply.com/products/hello-naturally-white...
[3]https://dailymed.nlm.nih.gov/dailymed/fda/fdaDrugXsl.cfm?set...
(Seems toms recently discontinued this, they mention it on their website, but say customers didn't like it)
[4]https://www.jason-personalcare.com/product/sea-fresh-anti-ca...
[5]https://popularitems.com/products/autobrush-kids-fluoride-fo...
As far as I can tell these are all real products and all meet the requirement of having fluoride and being SLS free.
Since you did return however and that was half my bet, I suppose you are still entitled to half my life savings. But the amount is small so maybe the knowledge of these new toothpastes is more valuable to you anyway.
If you want a correct answer the first time around, and give up if you don't get it, even if you know the thing can give it to you with a bit more effort (but still less effort than searching yourself), don't you think that's a user problem?
If you are genuinely asking a question, how are you supposed to know the first answer was incorrect?
I briefly got excited about the possibility of local LLMs as an offline knowledge base. Then I tried asking Gemma for a list of the tallest buildings in the world and it just made up a bunch. It even provided detailed information about the designers, year of construction etc.
I still hope it will get better. But I wonder if an LLM is the right tool for factual lookup - even if it is right, how do I know?
I wonder how quickly this will fall apart as LLM content proliferates. If it’s bad now, how bad will it be in a few years when there’s loads of false but credible LLM generated blogspam in the training data?
2 replies →
The person that started this conversation verified the answers were incorrect. So it sounds like you just do that. Check the results. If they turn out to be false, tell the LLM or make sure you're not on a bad one. It still likely to be faster than searching yourself.
14 replies →
This is the right question.
scientific method??
> don't you think that's a user problem?
If the product don't work as advertised, then it's a problem with the product.
I still remember when Altavista.digital and excite.com where brand new. They were revolutionary and very useful,even if they couldn't find results for all the prompts we made.
I am unconvinced that searching for this yourself is actually more effort than repeatedly asking the Mighty Oracle of Wrongness and cross-checking its utterances.