Comment by andrepd
4 days ago
The model has only a linguistic representation of what is "true" or "false"; you don't. This is a limitation of LLMs, human minds have more to it than NLP
4 days ago
The model has only a linguistic representation of what is "true" or "false"; you don't. This is a limitation of LLMs, human minds have more to it than NLP
LLMs are also more than NLP. They're deep learning models.
What? Yes the modelling technique falls under "deep learning" but it still very much processes language and language only, making it NLP.
Yes yes, language modelling ends up being surprisingly powerful at scale, but that doesn't make it not language modelling.
A couple of years of this LLM AI hype train has blinded people to what was actually surprising about LLMs. The surprise wasn't that you could make a language model and it wasn't that a language model could generate text. Those are both rather pedestrian observations, and their implementations are trivial. The surprise of LLMs was that contemporary hardware could scale this far and that an un-curated training set turns out to contain a statistically significant amount of truth. Deep learning was interesting because we didn't expect that amount of computation to be feasible at this time in human history, not because nobody had ever thought of it before.
The surprise of the LLM AI was that they were somewhat truthful at all.
3 replies →