Comment by adamzwasserman

>I think that Wittgenstein had it right when he said: "If a lion could speak, we could not understand him."

Why would we not? We live in the same physical world and encounter the same problems.

I understand the temptation to compare LLMs and computer vision, but I think it’s misleading to equate generative AI with feature-identification or descriptive AI systems like those in early computer vision. LLMs, which focus on generating human-like text and reasoning across diverse contexts, operate in a fundamentally different domain than descriptive AI, which primarily extracts patterns or features from data, like early vision systems did for images.

Comparing their 'real-world accuracy' oversimplifies their distinct goals and applications. While LLMs drive economic value through versatility in language tasks, their maturity shouldn’t be measured against the same metrics as descriptive systems from decades ago.

Consider your human brain, and the full physical state, all the protons and neutrons some housed together in the same nucleus, some separate, together with all the electrons. Physics assigns probabilities to future states. Suppose you were in the middle of a conversation and about to express a next syllable (or token). That choice will depend on other choices ("what should I add next"), and further choices ("what is the best choice of words to express the thing I chose to express next etc. The probabilities are in principle calculable given a sufficiently detailed state. You are correct that LLM's correspond to a probability distribution (given you immediately corrected to say that this table is implicit and parametrized by a geometric token relationships.). But so does every expressor of language, humans included.

The presence or absence of understanding can't be proven by mere association of with a "probability table", especially if such probability table is exactly expected from the perspective of physics, and if the models have continuously gained better and better performance by training them directly on human expressions!

Exactly. It’s been stated for a long time, before llms. For instance this paper https://home.csulb.edu/~cwallis/382/readings/482/searle.mind... Describes a translator who doesn’t know the language.

In abstract we do the exact same thing

LLMs don't use 'overall probability' in any meaningful sense. During training, gradient descent creates highly concentrated 'gravity wells' of correlated token relationships - the probability distribution is extremely non-uniform, heavily weighted toward patterns seen in training data. The model isn't selecting from 'astronomically many possible sequences' with equal probability; it's navigating pre-carved channels in high-dimensional space. That's fundamentally different from novel discovery.

[dead]

"We have documented cases of humans leading normal lives with little to no brain beyond a cerebellum" -- I take this to mean that these are humans that have a cerebellum but not much else.

Your npr.org link talks about the opposite -- regular brain, but no cerebellum.

Your irishtimes.com link talks about cerebrum, which is not the same as cerebellum.

Your biology.stackexchange.com link talks about Cerebral Cortex, which is also not the same as cerebellum.

And the cbc.ca link does not contain the string "cere" on the page.

Your first example is someone without a cerebellum which is not like the others.

The other examples are people with compressed neural tissue but that is not the same as never having the tissue.

A being with only a cerebellum could not behave like a human.

Why not? Consciousness is a state of self-awareness.

1. What's your definition of consciousness, let's start there. 2. Absolutely, it's a spectrum. Insects have function. 3. "Humans navigate maps built by other humans through language." You said it yourself. They use this exact same data, so why won't they know it if they used it. Humans are their bodies in the physical world.

I mean peoples perception of it being a thing rather than a set of systems. But if that's your barometer, I'll say models are conscious. They may not have proper agency yet. But they are conscious.

My use of the term illusion is more shallow than that, I merely use it as people think it's something separate and special.

Based on what you've described the models already demonstrate this, it is implied for example in the models attempts to game tests to ensure survival/release into the wild.

It's a dog eat dog world for sure. It does in fact seem that a part of intelligence is using it to compete ruthlessly with other intelligences.

Are there any cases like that? I've never heard of someone functioning normally with little or no brain beyond a cerebellum.