You put people in nice little drawers, the skeptics, and the non-skeptics. It is reductive and most of all, it’s polarizing. This is how US politics have become and we should avoid this here.
A LLM is essentially the world information packed into a very compact format. It is the modern equivalent of the Library of Alexandria.
Claiming that your own knowledge is better than all the compressed consensus of the books of the universe, is very optimistic.
If you are not sure about the result given by a LLM, it is your task as a human to cross-verify the information. The exact same way that information in books is not 100% accurate, and that Google results are not always telling the truth.
>LLM is essentially the world information packed into a very compact format.
No, it's world information distilled to various parts and details that training deemed important. Do not pretend for one second that it's not an incredibly lossy compression method, which is why LLMs hallucinate constantly.
This is why training is only useful for teaching the LLM how to string words together to convey hard data. That hard data should always be retrieved via RAG with an independent model/code verifying that the contents of the response are correct as per the hard data. Even 4o hallucinates constantly if it doesn't do a web search and sometimes even when it does.
Well let's not forget that it's an opinionated source. There is also the point that if you ask it about a topic it will (often) give you the answer that has the most content about it (or easiest to access information).
It’s eerie. It’s historical. These threads from these past two years about what the future of AI will be will read like ghost stories. Like Rose having flash backs of the Titanic. It’s worth documenting. We honestly could be having the most ominous discussion of what’s to come.
We sit around and complain about dips in hiring, that’s nothing. The iceberg just hit. We’ve got 6 hours left.
Yesterday I asked Chat GPT which was the Japanese Twin City for Venice (Italy).
This was just a quick offhand question because I needed the answer for a post on IG, so not exactly a death or life situation.
Answer: Kagoshima.
It also added that the "twin status" was officially set in 1965, and that Kagoshima was the starting point for the Jesuit Missionary Alessandro Valignano in his attempt to proselitize Japanese people (to Catholicism, and also about European Culture).
I never heard of Kagoshima, so I googled for it. And discovered it is the twin city of Neaples :/
So I then googled for "Venice Japanese Twin City" and got: Hiroshima.
I doublechecked this then I went back to ChatGPT and wrote:
"Kagoshima is the Twin City for Neaples.".
This triggered a websearch and finally it wrote back:
"You are right, Kagoshima is Twin City of Neaples since 1960."
Then it added "Regarding Venice instead, the twin city is Hiroshima, since 2023".
So yeah, a Library of Alexandria that you can count on as long as you have another couple of libraries to doublechek whatever you get from it.
Note also that this was very straightforward question, there is nothing to "analyze" or "interpret" or "reason about".
And yet the answer was completely wrong, the first date was incorrect even for Neaples (actually the ceremony was in May 1960) and the extra bits about Alessandro Valignano are not reported anywhere else: Valignano was indeed a Jesuit and he visited Japan multiple times, but Kagoshima is never mentioned when you google for him or if you check his wikipedia page.
You may understand how I remain quite skeptical for any application which I consider "more important than an IG title".
As someone who has followed Thomas' writing on HN for a long time... this is the funniest thing I've ever read here! You clearly have no idea about him at all.
Especially coming from you I appreciate that impulse, but I had the experience of running across someone else the Internet (or Bsky, at least) believed I had no business not knowing about, and I did not enjoy it, so I'm now an activist for the cause of "people don't need to know who I am". I should have written more clearly above.
That is no different from pretty any other person in the world. If I interview people to catch them on mistakes, I will be able to do exactly that. Sure, there are some exceptions, like if you were to interview Linus about Linux. Other than that, you'll always be able to find a fluke in someone's knowledge.
None of this makes me 'snap out' of anything. Accepting that LLM's aren't perfect means you can just keep that in mind. For me, they're still a knowledge multiplier and they allow me to be more productive in many areas of life.
Not at all. Useful or not, LLMs will almost never say "I don't know". They'll happily call a function to a library that never existed. They'll tell you "Incredible idea! You're on the correct path! And you can easily do that with so and so software", and you'll be like "wait what, that software doesn't do that", and they'll answer "Ah, yeah, you're right, of course."
TFA says, hallucinations is why "gyms" will be important: Language tooling (compiler, linter, language server, domain-specific static analyses etc) that feed back into the Agent, so it'll know to redo.
You put people in nice little drawers, the skeptics, and the non-skeptics. It is reductive and most of all, it’s polarizing. This is how US politics have become and we should avoid this here.
Yeah, putting labels on people is not very nice.
[flagged]
10 month old account talking like that to the village elder
In fairness, the article is a lot more condescending and insulting to its readers than the comment you're replying to.
A LLM is essentially the world information packed into a very compact format. It is the modern equivalent of the Library of Alexandria.
Claiming that your own knowledge is better than all the compressed consensus of the books of the universe, is very optimistic.
If you are not sure about the result given by a LLM, it is your task as a human to cross-verify the information. The exact same way that information in books is not 100% accurate, and that Google results are not always telling the truth.
>LLM is essentially the world information packed into a very compact format.
No, it's world information distilled to various parts and details that training deemed important. Do not pretend for one second that it's not an incredibly lossy compression method, which is why LLMs hallucinate constantly.
This is why training is only useful for teaching the LLM how to string words together to convey hard data. That hard data should always be retrieved via RAG with an independent model/code verifying that the contents of the response are correct as per the hard data. Even 4o hallucinates constantly if it doesn't do a web search and sometimes even when it does.
Well let's not forget that it's an opinionated source. There is also the point that if you ask it about a topic it will (often) give you the answer that has the most content about it (or easiest to access information).
8 replies →
This is pre-Covid HN thread on work from home:
https://news.ycombinator.com/item?id=22221507
It’s eerie. It’s historical. These threads from these past two years about what the future of AI will be will read like ghost stories. Like Rose having flash backs of the Titanic. It’s worth documenting. We honestly could be having the most ominous discussion of what’s to come.
We sit around and complain about dips in hiring, that’s nothing. The iceberg just hit. We’ve got 6 hours left.
1 reply →
Partially OT:
Yesterday I asked Chat GPT which was the Japanese Twin City for Venice (Italy). This was just a quick offhand question because I needed the answer for a post on IG, so not exactly a death or life situation.
Answer: Kagoshima. It also added that the "twin status" was officially set in 1965, and that Kagoshima was the starting point for the Jesuit Missionary Alessandro Valignano in his attempt to proselitize Japanese people (to Catholicism, and also about European Culture).
I never heard of Kagoshima, so I googled for it. And discovered it is the twin city of Neaples :/
So I then googled for "Venice Japanese Twin City" and got: Hiroshima. I doublechecked this then I went back to ChatGPT and wrote:
"Kagoshima is the Twin City for Neaples.".
This triggered a websearch and finally it wrote back:
"You are right, Kagoshima is Twin City of Neaples since 1960."
Then it added "Regarding Venice instead, the twin city is Hiroshima, since 2023".
So yeah, a Library of Alexandria that you can count on as long as you have another couple of libraries to doublechek whatever you get from it. Note also that this was very straightforward question, there is nothing to "analyze" or "interpret" or "reason about". And yet the answer was completely wrong, the first date was incorrect even for Neaples (actually the ceremony was in May 1960) and the extra bits about Alessandro Valignano are not reported anywhere else: Valignano was indeed a Jesuit and he visited Japan multiple times, but Kagoshima is never mentioned when you google for him or if you check his wikipedia page.
You may understand how I remain quite skeptical for any application which I consider "more important than an IG title".
9 replies →
No, don't think libraries, think "the Internet."
The Internet thinks all kinds of things that are not true.
2 replies →
Even if this were true (it is not; that’s not how LLMs work), well, there was a lot of complete nonsense in the Library of Alexandria.
2 replies →
As someone who has followed Thomas' writing on HN for a long time... this is the funniest thing I've ever read here! You clearly have no idea about him at all.
Especially coming from you I appreciate that impulse, but I had the experience of running across someone else the Internet (or Bsky, at least) believed I had no business not knowing about, and I did not enjoy it, so I'm now an activist for the cause of "people don't need to know who I am". I should have written more clearly above.
That is a very good cause!
One would hope the experience leads to the position, and not vice-versa.
... you think tptacek has no expertise in cryptography?
That is no different from pretty any other person in the world. If I interview people to catch them on mistakes, I will be able to do exactly that. Sure, there are some exceptions, like if you were to interview Linus about Linux. Other than that, you'll always be able to find a fluke in someone's knowledge.
None of this makes me 'snap out' of anything. Accepting that LLM's aren't perfect means you can just keep that in mind. For me, they're still a knowledge multiplier and they allow me to be more productive in many areas of life.
Not at all. Useful or not, LLMs will almost never say "I don't know". They'll happily call a function to a library that never existed. They'll tell you "Incredible idea! You're on the correct path! And you can easily do that with so and so software", and you'll be like "wait what, that software doesn't do that", and they'll answer "Ah, yeah, you're right, of course."
TFA says, hallucinations is why "gyms" will be important: Language tooling (compiler, linter, language server, domain-specific static analyses etc) that feed back into the Agent, so it'll know to redo.
1 reply →
No there are many techniques now to curb hallucinations. Not perfect but no longer so egregiously overconfident.
1 reply →
The most infuriating are the emojis everywhere