Comment by ninetyninenine
14 days ago
>Calling it a token-predictor isn't reductionism. It's designed, implemented and trained for token prediction. Training means that the weights are adjusted in the network until it accurately predicts tokens. Predicting a token is something along the lines of removing a word from a sentence and getting it to predict it back: "The quick brown fox jumped over the lazy ____". Correct prediction is "dogs".
It absolutely is reductionism. Ask any expert who knows how these things work and they will say the same:
https://youtu.be/qrvK_KuIeJk?t=497
Above we have Geoffrey Hinton, the godfather of the current wave of AI saying your statements are absolutely crazy.
It's nuts that I don't actually have to offer any proof to convince you. Proof won't convince you. I just have to show you someone smarter than you with a better reputation saying the exact opposite and that is what flips you.
Human psychology readily can attack logic and rationality. You can scaffold any amount of twisted logic and irrelevant analogies to get around any bulwark to support your own point. Human psychology fails when attacking another person who has a higher rank. Going against someone of higher rank this causes you to think twice and rethink your own position. In debates, logic is ineffective, bringing opposing statements with (while offering ZERO concrete evidence) from experts with a higher rank is the actual way to convince you.
>But, oh, just believe the LLM when it produces a sentence referring to itself, claiming it is conscious.
This is an hallucination. Showing that you're not much different from an LLM. I NEVER stated this. I said it's possible. But I said we cannot make a definitive statement either way. We cannot say it isn't conscious we cannot say it is. First we don't understand the LLM and second WE don't even have an exact definition of consciousness. So to say it's not conscious is JUST as ludicrous as saying it is.
Understand?
> the godfather of the current wave of AI
Here we go again...
Geoffrey Hinton is popular because of ML. You know why he doesn't understand the modern wave of LLMs? Because the transformer architecture isn't his invention. He had almost nothing to do with it. It's like saying that nobody understands how the nuclear bomb works simply because Einstein won't explain it to you in detail. There is a science here, and denying that it can be interpreted or understood is simply hysterical. We can perfectly well understand a Markov chain, why are LLMs any different?
The "current wave of AI" is an autoregressive feed-forward algorithm inferring tokenized data using complex pretrained weights. In simple terms, you could describe this as a token predictor without glossing over any details that are fundamentally important. To call it "reductionist" you would have to explain what is missing from the definition, not simply that you distrust people who understand it.
Unless I'm mistaken, the transformer architecture isn't your invention either, yet you claim to understand it better than Geoffrey Hinton.
Did you invent the transformer architecture yourself, and I am mistaken to guess that you probably understand it less than Geoffrey Hinton does?
I'll venture a wild guess that you probably haven't been working in the field as long as he has, and you don't personally know as many people in the field as he does, huh?
Or is my presumptuous chain of logic just a random thoughtless unconscious probabilistic stream of tokens that has nothing to do with reality, and I don't realize I'm speaking to Ashish Vaswani himself?
Then why don't you use your real name as you HN handle, are you really that modest, Ashish? And don't you have better things to do than posting to Hacker News? If you really wanted to win an argument from authority, you could at least post under your real name, Mr. Vaswani.
[flagged]