Comment by semiquaver
4 days ago
Do you not say your words one-at-a-time like everyone else? Otherwise I can’t see how my comment is “wrong”
4 days ago
Do you not say your words one-at-a-time like everyone else? Otherwise I can’t see how my comment is “wrong”
Even if you could understand human cognition to the level required to say, confidently, that it’s done one word at a time, it’s likely not! Natural language is not a prerequisite for human intelligence, as evidenced by the fact that we went from primates to commenting on HN.
Natural language is, however, a prerequisite for the existence of LLMs. It’s more similar to methods for storing and retrieving information, like the printing press or a database, than it is to a sentient being.
That’s not to say that LLMs can’t do crazy things, because they already have. Our language can encode a whole lot of information, and it’s incredible that we’ve found a way to distill that so effectively.
Even if you could understand human cognition to the level required to say, confidently, that it’s done one word at a time, it’s likely not!
I think they’re not talking about cognition, but about output: regardless of what may be happening inside your brain, ultimately one word at a time comes out of your mouth, right? And you can’t then unsay it.
When you put it in those terms, LLMs are in exactly the same boat.
Deepseek zero didn’t mix up all languages in something very efficient?
Interesting thought but I assume a lot of samples in the training corpus are examples of translation between languages and the same text in different languages.
Only one word at a time!?! It's time you embrace the way of the diffusion model and hazily refine your entire thought until it's coherent.
> Do you not say your words one-at-a-time like everyone else
You're conflating being autoregressive with being sequential.