Comment by HarHarVeryFunny
16 hours ago
It's been a long time since I read Hofstader's book "Fluid Concepts and Creative Analogies" that this is presumably based on, so I'll be interested to watch the video as a refresher, later. :) He also wrote a follow-on book, "Surfaces and Essences: Analogy as the Fuel and Fire of Thinking", published after this video was made, that I've never read.
From what I recall, to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process all the way from perception on up, and is the mechanism by which we form object categories in the first place.
As always Hofstader's ideas are interesting, but I can't say I agree with them. It seems that the key evolutionary benefit, and function, of a brain is prediction, which is the superpower that moves us from being stuck in the present to being able to "see" (predict) the future, and therefore from being merely reactive to being able to proactively plan and predict future outcomes (what will the sabre-tooth do, where is the water supply?) based on our experience.
Given the never-same-twice nature of sensory perception, before you can predict you need to be able to generalize/categorize, which I think Hofstader would regarded as analogy making (how is this thing I'm seeing similar to what I've previously seen?), although it seems the actual mechanism involved is embeddings or embedding-like representations where similar inputs have similar representations, and what might more simply be considered as associative recall provides the generalization from view/instance to identity/category.
So, is it really analogies all the way up, or are our perception and cognitive processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations, but also match the evolutionary needs that we would expect to exist?
When my daughter was 6 months old, she once pointed at a towel and said "P". Apparently she recognize Winnie the Pooh from a book she saw earlier.
If you think about it, this requires recognizing something common in things which are very different: e.g. book and towel have different texture and color, and you somehow need to separate what's depicted from the background. So that's pretty much innate, core brain function. I mean, any animal with vision can recognize an object from the background - otherwise vision is useless. But for humans (and some animals) this translates to depiction of object on a flat surface very easily.
So, yeah, analogies-all-the-way-down seems plausible. Even object-vs-background and depiction-vs-paper is itself an analogy.
Well, recognizing (recalling) Pooh on a towel as being the same as a picture of Pooh in book can also just be regarded as generalization, same as recognizing that someone looking left is the same person as them looking right, etc.
Our cortex (all of it - visual, auditory, association. motor..) is remarkably simple and regular - it's six layers of neurons with a specific circuit design, that appears to make it a predictor, with top down predictions meeting bottom up sensory feedback. As noted, it makes a ton of sense that evolution would have equipped us with a predictor (and the more powerful the better). Prediction directly supports reasoning and planning (multi-step what-if prediction).
So, my question for Hofstader, or you, would be given that the brain seems to be literally built for prediction, then what is to be gained by regarding this as analogy making instead? Is there any evidence for this? Even as a philosophical viewpoint, doesn't that make for very convoluted explanations of simple things like reasoning and language? (I explained prediction-based reasoning above in one sentence - the analogy-based explanation takes a whole dense book!).
One could certainly use the language of analogies to describe perception (Pooh's ears on the towel are analogous to his ears in the book, etc), but what is the benefit of doing so, rather than the simpler view of this just being generalization (which is specifically what the animal needs), and where is the evidence in the brain?
> to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process
This is so funny to me, because as many people know, sharing an analogy with another person is the fastest way to LOSE an argument with someone, or otherwise spiral it into an unproductive place.
I think it’s Scott Adams who used to say analogies work well for explaining. They work terribly for persuasion.
> sharing an analogy with another person is the fastest way to LOSE an argument with someone
This is true if the arguer is hostile, but as I've gotten older, if I get the sense that someone is entering an argument with the primary goal of "winning", I'll try to avoid that framing or just look for an offramp entirely.
Sure, the other party might think they "won", but they were going to think that anyway. For those more inclined to feel that the point of talking to each other is to learn from each other, I'll continue to use analogies and other things reasonable people understand.
> This is true if the arguer is hostile, but as I've gotten older, if I get the sense that someone is entering an argument with the primary goal of "winning", I'll try to avoid that framing or just look for an offramp entirely.
Absolutely. The best kinds of arguments are those where you both have the shared goal of reaching consensus, and treat reaching consensus as a collaborative activity of finding the correct answer, even though you disagree with the starting point. There are a few techniques for this, which work well when operating in good faith, and can backfire when dealing with hostile counterparties. Most things work badly when arguing with hostile counterparties.
> They work terribly for persuasion.
I just experienced this in a conversation. An analogy offers an opportunity to engage with the straw man and miss the forest for the trees.
I think that's just because every analogy gives an entire "second front" of ideas for a hostile recipient to find a "flaw", when they ignore the intended boundary between the stuff that does/doesn't matter to the analogy.
Ex:
Explainer: "Getting a spleen means cutting open the patient and taking it out. It's just like how I'm going to unzip this section of the patient-shaped doll, and remove this little purple bean. In both cases a hole is necessary in a similar location."
Hostile listener: "Nonsense! I can just buy beans at the store! So just buy a spleen! No hole!"
Constructing explicit analogies for persuasion seems to be a bit of a different thing.
I’m not sure if Hofstadter puts it this way, but to me even the core aspects of your sentences in this post have roots in analogies. What does it mean to lose an argument or to spiral it to a different place? There is no place, there is no lost item, but we talk about these abstract ideas in ways that largely depend upon understanding things like physical objects and space and movement.
Ironically, I think he was right! In fact some of his initial experiments, like copycat, were about predicting patterns. You can see next-token-prediction from there. But I think he always held out for an algorithmic/logical method rather than a purely statistical one.
If he had accepted the "Bitter Lesson", I think he would have been at the forefront of LLMs.
Maybe you haven't noticed that the "Bitter Lesson" had itself a "Bitter Lesson" - that scaling pure data and compute did not lead to AGI: diminishing training returns, GPT-5 disappointment, even openAI stating it was the last 'pure scale' model.
The path forward all big llm providers ("ai" labs) have gone is neuro-symbolic (even though they publicly would never labeled it as such to not admit critics like Gary Marcus were right - even though all their actions actually point in that direction).
Neuro symbolic, rly? Can you please elaborate what it is that made you conclude that?
2 replies →
Yes, in the text he explicitly argues for categorization and analogizing being two sides of the same coin and functionally equivalent. For instance - what is an ‘embedding’? In itself its an analogy. Lakoff explored similar ground.
> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations
If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.)
Hofstadter referenced this back in the day as a promising beginning: https://en.wikipedia.org/wiki/Sparse_distributed_memory which sounds kind of similar in style to the embeddings you bring up.