Comment by trgn
1 day ago
i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.
in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.
maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.
Indeed. Who'd have thought that mere statistics could do that? Hat's off to them - they made the darn thing work.
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
https://babylm.github.io/
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.
"Mere statistics" is how physical reality works; I don't mean the language, the syntax
I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere
The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement
Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.
The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs
“Mere statistics” is heavily underselling what the transformer architecture does. Unless you think “mere ion channels” is a sensible way to think about the human brain.
> stochastic generation of language yields a form of recognizable intelligence.
Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".
My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.
And language itself is so very 'stochastic' too, due to complex subjectivity. Every word floats on a variable sea of individual choice.
For example, if I compose your post as :
--
"Good prose embodies intellect gradually evolved by myriad humans over time, so it wows me to see LLMs create it, though I soon realised that the intelligence is not being created by the LLM but rather is already present in the human-evolved input datasets - the prose itself! - which they statistically arrange."
--
..the particulars of the language are entirely different (and omit some of your chosen nuance) but most readers would receive the same "core truth" from it, the same measure of beneficial information.
Which highlights the great fluidity of language; that is another of the key underlying mechanisms allowing LLMs to flourish.
I think there is a distinction to draw there between stochasticity in the use and in the evolution of language (which is not clean as they are not independent). The language evolves stochastically, as it does so by being used in exchanges. But the use could be deterministic with the exchanges being random and the function of update based on exchange. I still think there is stochasticity in the use itself, I choose my words, go back and delete and rewrite, explain again in a different way when the audience looks at me unimpressed etc. I just don’t see fluidity as a proof of stochasticity of evolution or the other way around and in fact think the interaction is a quite interesting domain.
> i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.
It's funny, I used to think it was complete and utter horseshit when literature professors would say that the meaning of a text is created by the reader, not the author.
Since LLMs have become commonplace, I have completely reversed my opinion on that.
That you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it. It is why "claudish" is so hard to read--you are trying to impose meaning on a sequence of words that was stochastically generated, and it takes mental effort to do that.
I'm not so sure about that.
If Claude can write formal mathematical proofs (that a machine can check), or it can write software that works, where's the head of the reader that imposes the meaning?
i am not an expert in any of this, but intuitively i largely feel the same... the character we see as "claude" is mostly in our heads; its almost like a "super rubber duck" phenomenon or something
I fail to see what exactly is explained, or clarified by the theory here. Postmodernist philosophers are notorious for writing texts which make readers struggle. Surely, it doesn't mean "Foucault-ese" is somehow more stochastically generated than "Claudese".
Reading what you wrote, I must interpretively attempt to construct a model of what you may have meant and what you may believe, because on the surface it contains multiple claims that are frankly false in my experience (itself a probabilistic world model).
> a sequence of words that was stochastically generated
As was your comment (and mine, and the others on this page and elsewhere).
With Claude I know that there is no source meaning or belief, but it helps to impose Dennett's "intentional stance" and treat it as if there were, up to a point. That there is such a "point" allows be to manipulate it in my attempts to extract factual information implicit in its training data, expressed probabilistically via its stochastic processes.
The existence of LLMs is telling us something, surely it means something. I'm not sure we've quite figured it out yet but its probably something along the lines of 'intelligence might not be as complicated as we thought' ... a fourth narcissistic blow to humanity after Copernicus, Darwin and Freud.
Freud doesn't belong in that group. Freud was a fraud.
We train them on the sum of human knowledge available in digital format, so I'm not sure that's an indictment, and the large models are incredibly complicated to the point it's hard to reverse engineer how they work so well and what they're actually doing to build internal models to predict the next token. Several billion to several trillion parameters is anything but simple.
Right, we needed compute on a large and fast scale to make LLMs actually work. But the basic principles are quite simple: train on a huge dataset for long enough, using some fairly simple ML maths but across a very large dimensional space, and then out the other side you get something that can competently use language, and is capable of doing useful work. This is somewhat surprising, no?
1 reply →
The narcissism might not be in thinking 'intelligence might not be as complicated as we thought’ - the narcissism might be in thinking ‘LLM intelligence is the way we thought’.
I understand the charge, but it's soemwhat unfair. It's more like we start to see a correspondence which even 20 years ago was barely considered
3 replies →
> it's uncanny that stochastic generation of language yields a form of recognizable intelligence.
It most certainly does not. It's obvious to anyone who critically examines LLMs, without letting the hype override sense, that there's zero intelligence there.