Comment by akie
18 hours ago
That's a very concise and illuminating way to think about what's happening, IF (and only if) you already know how these models work. Thanks for that.
18 hours ago
That's a very concise and illuminating way to think about what's happening, IF (and only if) you already know how these models work. Thanks for that.
Yes this is more like compression to remember and not for learning/understanding.
Compression is the reason why these Models are able to learn and understand.
My brain is doing the exact same thing.
I learned enough to compress concepts like a bike and what a bike does and for what i can use a bike.
Ask a LLM and it will answer you similiar to humans.
Blind people learn concepts of bikes too and in a smiliar way: by description.
LLMs just have so much data in form of text available and are able to ingest all of this, that the LLM compression algorithm doesn't has to be that good/finetuned than ours.
But I would assume that Yann LeCun's JEPA or other breakthroughs in the next few years will get us there.
Compression and existence of mechanism to expound on it does not imply consciousness.
Otherwise, yes, finally people observe the very apparent fact that LLMs are one very smart compression.
> Blind people learn concepts of bikes too and in a smiliar way: by description.
And by touch and sound. And maybe some were daring enough to drive one, or unlucky enough to get hit by one. But have way more input than just texts.
9 replies →