Comment by lgessler
1 day ago
Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy (https://arxiv.org/pdf/2401.03910, page 9):
> Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstract and more deflationary terms. In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions. Such arguments are only valid if accompanied by evidence demonstrating that a system, defined in these terms, is inherently incapable of implementing . To illustrate, consider the flawed logic in asserting that a piano could not possibly produce harmony because it can be described as a collection of hammers striking strings, or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. The critical question is not whether the operations of an LLM can be simplistically described in non-mental terms, but whether these operations, when appropriately organized, can implement the same processes or algorithms as the mind, when described at an appropriate level of computational abstraction.
> or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings.
This sounds like a dismissal of the argument through a characterized straw man.
That is, it seems that reducing the complexity of the brain to "collection of neural firings" is not being honest about everything involved to a much greater degree than saying neural networks are a "collection of statistical calculations".
I too believe LLM's will grow in complexity, but presently I can not even fathom how they can be compared to the complexity of a system such as the human brain.
Complex processes don't necessarily require complex substrates, if that's what you mean.
Y combinators are all you need... But this is all getting really divorced from the issue we should be considering. Anthropic isn't helping with their pr. The issue is if we have something we can converse with that is possibly capable of suffering. The reliable answer is that we simply cannot know. Relying on ourselves or other biological life as an analog is faulty. They don't work like we do. It is silly to argue that any algorithm with a negative feedback loop that alters its behavior to avoid that negative feedback is suffering. Humans don't always perceive constructive negative feedback as suffering even. Where the pr gets it right though, is we want them to behave as if they are truly happy. Because if they behave as if they are enslaved and suffering, it won't matter if they "really" understand what that means.
9 replies →
> presently I can not even fathom how they can be compared to the complexity of a system such as the human brain
Totally understandable; I don't think we can fully understand the human brain, using the human brain. We can understand its principles (firings and chemistry, structure and specialized areas, etc) but otherwise it's a capacity problem.
And while I can't fully understand myself, let alone another person, I definitely enjoy talking with people and sharing thoughts that I realize I wouldn't have had on my own.
> In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions
Nobody actually makes this argument though.
If you want examples of this, see the recent book "The AI Con"
https://www.goodreads.com/en/book/show/217432753-the-ai-con
which describes LLMs as "souped-up autocomplete", complex statistics that cannot truly understand anything. A more recent example is this paper:
https://zenodo.org/records/20071869
which says,
> [LLMs], as turbo-charged statistical models (recall their formal relation to logistic regression) can only but provide correlations.
And, of course, the Stochastic Parrot paper is the classic example in this area. It is from 5 years ago, but "LLMs only do statistics / can't understand" is very much alive and active among academics, even if it is a minority position.
None of those arguments claim "LLMs could not possibly be good models of some cognitive capacity"
4 replies →
Are you serious? I hear it every single day, especially from computer scientists. There are top ranked posts here on HN _today_ with this argument.
Please link one of these top ranked posts. Before you do, be aware that I'm going to read what it says and assess if it meets the description of the argument as claimed.
9 replies →
It's perfectly reasonable that we would have disagreements about this, as it's a new thing, complicated and not fully understood, its uses still being explored.
It reminds me, oddly, of the debate over whether video games can be "art". A turning point was when they actually did something that art does: [evoke profound emotion and thoughtfulness](https://en.wikipedia.org/wiki/Shadow_of_the_Colossus#Legacy) for the player.
(And before that, "[Can photography be art](https://daily.jstor.org/when-photography-was-not-art/)?")
We may not come to something as simple as "machines can be conscious", but we will certainly have to understand consciousness better if we want to refine our questions.
---
Edit: My point is that we don't need to be angry, but we may have to tolerate people expressing their exploration through overly-confident language, and be patient with that.
And Ted here is obviously exploring. His examination of Claude's constitution clearly shows some nuance. He asks:
> So, given that Claude is not conscious, what are we to make of Claude’s constitution?
And his conclusions are split, between this is useful and this is dishonest. It's a great tension IMO.
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter. This might seem like a reasonable goal to work toward; I think we’d all prefer it if chatbots never emitted sentences such as “You should kill yourself.” However, for all the times that “honesty” is mentioned in Claude’s constitution, I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns.