Comment by hodgehog11
1 day ago
Are you serious? I hear it every single day, especially from computer scientists. There are top ranked posts here on HN _today_ with this argument.
1 day ago
Are you serious? I hear it every single day, especially from computer scientists. There are top ranked posts here on HN _today_ with this argument.
Please link one of these top ranked posts. Before you do, be aware that I'm going to read what it says and assess if it meets the description of the argument as claimed.
I understood the quoted sentence to be saying, in essence "people claim LLMs aren't really and can't really be thinking or experiencing anything" which is certainly something people say and have written papers on.
The phenomenological quality of subjective experience is never described as "cognitive capacity".
That term is used to describe mental aptitude or skills, like the ability to learn new languages or do math.
1 reply →
As an example, "They're made out of weights" describes why the weight-based construction of neural networks should impact the way that you think about them and their outputs. I would argue that an offhand description of its microscopic formulation tells us nothing at all about how to think about these outputs, or the models themselves. Even if it is a cute story, I think it definitely classifies as succumbing to this fallacy, but maybe I missed some subtle point that you or someone would be happy to illuminate?
By the way, I know it's a parody of another story that makes this exact refutation. But I think this only serves to highlight the point.
> They're made out of weights" describes why the weight-based construction of neural networks should impact the way that you think about them and their outputs.
How do you connect that description to "LLMs could not possibly be good models of some cognitive capacity"?
4 replies →
It's perfectly reasonable that we would have disagreements about this, as it's a new thing, complicated and not fully understood, its uses still being explored.
It reminds me, oddly, of the debate over whether video games can be "art". A turning point was when they actually did something that art does: [evoke profound emotion and thoughtfulness](https://en.wikipedia.org/wiki/Shadow_of_the_Colossus#Legacy) for the player.
(And before that, "[Can photography be art](https://daily.jstor.org/when-photography-was-not-art/)?")
We may not come to something as simple as "machines can be conscious", but we will certainly have to understand consciousness better if we want to refine our questions.
---
Edit: My point is that we don't need to be angry, but we may have to tolerate people expressing their exploration through overly-confident language, and be patient with that.
And Ted here is obviously exploring. His examination of Claude's constitution clearly shows some nuance. He asks:
> So, given that Claude is not conscious, what are we to make of Claude’s constitution?
And his conclusions are split, between this is useful and this is dishonest. It's a great tension IMO.
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter. This might seem like a reasonable goal to work toward; I think we’d all prefer it if chatbots never emitted sentences such as “You should kill yourself.” However, for all the times that “honesty” is mentioned in Claude’s constitution, I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns.