Comment by RayVR
3 days ago
Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool.
My personal assessment is that LLMs can do neither.
Words are the "simplistic" projection of an LLM's abstract thoughts.
An LLM has: words in its input plane, words in its output plane, and A LOT of cross-linked internals between the two.
Those internals aren't "words" at all - and it's where most of the "action" happens. It's how LLMs can do things like translate from language to language, or recall knowledge they only encountered in English in the training data while speaking German.
> It's how LLMs can do things like translate from language to language
The heavy lifting here is done by embeddings. This does not require a world model or “thought”.
LLMs are compression and prediction. The most efficient way to (lossfully) compress most things is by actually understanding them. Not saying LLMs are doing a good job of that, but that is the fundamental mechanism here.
3 replies →
The "cross-linked internals" only go one direction and only one token at a time, slide window and repeat. The RL layer then picks which few sequences of words are best based on human feedback in a single step. Even "thinking" is just doing this in a loop with a "think" token. It is such a ridiculously simplistic model that it is vastly closer to an adder than a human brain.
I'm definitely a stream of words.
My "abstract thoughts" are a stream of words too, they just don't get sounded out.
Tbf I'd rather they weren't there in the first place.
But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive.
Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects an answer along the lines of dying for one's nation or some shit. Instead, the soldier replies: "Well, to see what happens next!"
I doubt words are involved when we e.g. solve a mathematical problem.
To me, solving problems happens in a logico/aesthetical space which may be the same as when you are intellectually affected by a work of art. I don't remember myself being able to translate directly into words what I feel for a great movie or piece of music, even if in the late I can translate this "complex mental entity" into words, exactly like I can tell to someone how we need to change the architecture of a program in order to solve something after having looked up and right for a few seconds.
It seems to me that we have an inner system that is much faster than language, that creates entities that can then beslowly and sometimes painfully translated to language.
I do note that I'm not sure about any of the previous statements though'
My wordmangling and mathsolving happen in that sort of logico/aesthetical space, too!
The twist about words in particular is they are distinctly articulable symbols, i.e. you can sound 'em out - and thus, presumably, have a reasonable expectation for bearers of the same language to comprehend if not what you meant then at least some vaguely predictable meaning-cloud associated with the given speech act.
That's unlike e.g. the numbers (which are more compressed, and thus easier to get wrong), or the syntagms of a programming language (which don't even have a canonical sonic representation).
Therefore, it's usually words that are taught to a mind during the formative stages of its emergence. That is, the words that you are taught, your means of inner reflection, are still sort of an imposition from the outside.
Just consider what you life trajectory would've been if in your childhood you had refused to learn any words, or learned them and then refused to mistake them for the things they represent!
Infants and even some animals recognize their reflection in a mirror; however, practically speaking, introspection is something that one needs to be taught: after recognizing your reflection you still need to be instructed what is to be done about it.
Unfortunately, introspection needing to be taught means that introspection can be taught wrongly.
As you can see with the archetypical case of "old and wise person does something completely stupid in response to communication via digital device", a common failure mode of how people are taught introspection (and, I figure, an intentional one!) is not being able to tell apart yourself from your self, i.e. not having an intuitive sense of where the boundary lies between perception and cognition, i.e. going through life without ever learning the difference between the "you" and the "words about you".
It's extremely common, and IMO an extremely factory-farming kind of tragic.
I say it must be extremely intentional as well, because the well-known practice of using "introspection modulators" to establish some sort of perceptual point of reference (such as where the interior logicoaeshtetical space ends and exterior causalityspace begins) very often ends up with the user in, well, a cage of some sort.
1 reply →
<< My "abstract thoughts" are a stream of words too, they just don't get sounded out.
Hmm, seems unlikely. They are not sounded out part is true, sure, but I question whether 'abstract thoughts' can be so easily dismissed as mere words.
edit: come to think of it and I am asking this for a reason: do you hear your abstract thoughts?
Different people have different levels of internal monologuing or none at all. I don't generally think with words in sentences in my head, but many people I know do.
2 replies →
>do you hear your abstract thoughts?
Most of the fucking time, and I would prefer that I didn't. I even wrote that, lol.
I don't think they're really "mine", either. It's just all the stuff I heard somewhere, coalescing into potential verbalizations in response to perceiving my surroundings or introspecting my memory.
If you are a materialist positivist, well sure, the process underlying all that is some bunch of neural activation patterns or whatever; the words remain the qualia in which that process is available to my perception.
It's all cuz I grew up in a cargo cult - where not presenting the correct passwords would result in denial of sustenance, shelter, and eventually bodily integrity. While presenting the correct passwords had sufficient intimidation value to advance one's movement towards the "mock airbase" (i.e. the feeder and/or pleasure center activation button as provided during the given timeframe).
Furthermore - regardless whether I've been historically afforded any sort of choice in how to conceptualize my own thought processes, or indeed whether to have those in the first place - any entity which has actual power to determine my state of existence (think institutions, businesses, gangs, particularly capable individuals - all sorts of autonomous corpora) has no choice but to interpret me as either a sequence of words, a sequence of numbers, or some other symbol sequence (e.g. the ones printed on my identity documents, the ones recorded in my bank's database, or the metadata gathered from my online represence).
My first-person perspective, being constitutionally inaccessible to such entities, does not have practical significance to them, and is thus elided from the process of "self-determination". As far as anyone's concerned, "I" am a particular sequence of that anyone's preferred representational symbols. For example if you relate to me on the personal level, I will probably be a sequence of your emotions. Either way, what I may hypothetically be to myself is practically immaterial and therefore not a valid object of communication.
Then what are non-human animals doing?
Excellent point. See also the failure of Sapir-Whorf to prove that language determines thought. I think we have plenty of evidence that, while language can influence thought, it is not thought itself. Many people invested in AI are happy to throw out decades of linguistic evidence that language and thought are separate.
1 reply →
Living and dying - and also, when humans are involved, being used.
Even if they are "simplistic projections", which I don't think is the correct way to think about it, there's no reason that more LLM thoughts in middle layers can't also exist and project down at the end. Though there might be efficency issues because the latent thoughts have to be recomputed a lot.
Though I do think in human brains it's also an interplay where what we write/say also loops back into the thinking as well. Which is something which is efficient for LLMs.
I am a stream of words - I have even ran out of tokens while speaking before :)
But raising kids, I can clearly see that intelligence isn't just solved by LLMs
> But raising kids, I can clearly see that intelligence isn't just solved by LLMs
Funny, I have the opposite experience. Like early LLMs kids tend to give specific answers to the questions they don't understand or don't really know or remember the answer to. Kids also loop (give the same reply repeatedly to different prompts), enter highly emotional states where their output is garbled (everyone loves that one), etc. And it seems impossible to correct these until they just get smarter as their brain grows.
What's even more funny is that adults tend to do all these things as well, just less often.
Same failure modes, but not a general solution to intelligence.
2 replies →
LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?
If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.
Human brains aren’t magic in the literal sense but do have a lot of mechanisms we don’t understand.
They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities.
They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability.
I definitely agree with the materialist view that we will ultimately be able to emulate the brain using computation but we’re nowhere near that yet nor should we undersell the complexity involved.
When someone says "AIs aren't really thinking" because AIs don't think like people do, what I hear is "Airplanes aren't really flying" because airplanes don't fly like birds do.
13 replies →
I agree we shouldn't undersell or underestimate the complexity involved, but when LLM's start contributing significant ideas to scientists and mathematicians, its time to recognize that whatever tricks are used in biology (humans, octopuses, ...) may still be of interest and of value, but they no longer seem like the unique magical missing ingredients which were so long sought after.
From this point on its all about efficiencies:
modeling efficiency: how do we best fit the elephant, with bezier curves, rational polynomials, ...?
memory bandwidth training efficiency: when building coincidence statistics, say bigrams, is it really necessary to update the weights for all concepts? a co-occurence of 2 concepts should just increase the predicted probability for the just observed bigram and then decrease a global coefficient used to scale the predicted probabilities. I.e. observing a baobab tree + an elephant in the same image/sentence/... should not change the relative probabilities of observing french fries + milkshake versus bicycle + windmill. This indicates different architectures should be possible with much lower training costs, by only updating weights of the concepts observed in the last bigram.
and so on with all other kinds of efficiencies.
ofc, and probably will never understand because of sheer complexity. It doesn't mean we can't replicate the output distribution through data. Probably when we do in efficient manners, the mechanisms (if they are efficient) will be learned too.
DNA inside neurons uses superconductive quantum computations [1].
[1] https://www.nature.com/articles/s41598-024-62539-5
As the result, all living cells with DNA emit coherent (as in lasers) light [2]. There is a theory that this light also facilitates intercellular communication.
[2] https://www.sciencealert.com/we-emit-a-visible-light-that-va...
Chemical structures in dendrites, not even neurons, are capable to compute XOR [3] which require multilevel artificial neural network with at least 9 parameters. Some neurons in brain have hundredths of thousands of dendrites, we are now talking of millions of parameters only in single neuron's dendrites functionality.
[3] https://www.science.org/doi/10.1126/science.aax6239
So, while human brains aren't magic, special or different, they are just extremely complex.
Imagine building a computer with 85 billions of superconducting quantum computers, optically and electrically connected, each capable of performing computations of a non-negligibly complex artificial neural network.
All three appear to be technically correct, but are (normally) only incidental to the operation of neurons as neurons. We know this because we can test what aspects of neurons actually lead to practical real world effects. Neurophysiology is not a particularly obscure or occult field, so there are many many papers and textbooks on the topic.(And there's a large subset you can test on yourself, besides, though I wouldn't recommend patch-clamping!)
2 replies →
They are extremely complex, but is that complexity required for building a thinking machine? We don't understand bird physiology enough to build a bird from scratch, but an airplane flies just the same.
3 replies →
(Motors and human brains are both just mechanisms, the reason one is a priori capable of learning abstract thought and not the other ?)
While I agree to some extent with the materialistic conception, the brain is not an isolated mechanism, but rather the element of a system which itself isn't isolated from the experience of being a body in a world interacting with different systems to form super systems.
The brain must be a very efficient mechanism, because it doesn't need to ingest the whole textual production of the human world in order to know how to write masterpieces (music, litterature, films, software, theorems etc...). Instead the brain learns to be this very efficient mechanism with (as a starting process) feeling its own body sh*t on itself during a long part of its childhood.
I can teach someone to become really good at producing fine and efficient software, but on the contrary I can only observe everyday that my LLM of choice keeps being stupid even when I explain it how it fails. ("You're perfectly right !").
It is true that there's nothing magical about the brain, but I am pretty sure it must be stronger tech than a probabilistic/statistical next word guesser (otherwise there would be much more consensus about the usability of LLMs I think).
[dead]
I'm not arguing that human brains are magic. the current AI models will probably teach us more about what we didn't know about intelligence than anything else.
Right, I'm just going to teach my dog to do my job then and get free money as my brain is no more magic, special or different to theirs!
There isn't anything else around quite like a human brain that we know of, so yes, I'd say they're special and different.
Animals and computers come close in some ways but aren't quite there.
For some unexplainable reason your subjective experience happens to be localized in your brain. Sounds pretty special to me.
There is nothing special about that either. LLM's also have self awareness/introspection, or at least a some version of it.
https://www.anthropic.com/research/introspection
Its hard to tell sometimes because we specifically train them to believe they don't.
1 reply →
> LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?
“Internal combustion engines and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?”
The question isn't about what an hypothetical mechanism can do or not, it's about whether the concrete mechanism we built does or not. And this one doesn't.
The general argument you make is correct, but you conclusion "And this one doesn't." is as yet uncertain.
I will absolutely say that all ML methods known are literally too stupid to live, as in no living thing can get away with making so many mistakes before it's learned anything, but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished.
What is "abstract thought"? Is that even the same between any two humans who use that word to describe their own inner processes? Because "imagination"/"visualise" certainly isn't.
18 replies →
Thermometers and human brains are both mechanisms. Why would one be capable of measuring temperature and other capable of learning abstract thought?
> If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.
Google "strawman".