Comment by why_at
19 hours ago
I was having a hard time pinning down what bothered me about this but I think you put it pretty well.
It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were.
But it doesn't really give a reason to accept the analogy, it just asserts it.
There's a big difference between a whole civilization and a piece of software that can output text.
>But it doesn't really give a reason to accept the analogy, it just asserts it.
It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.
This. I think a lot of people discussing it here are missing this point.
The bit that lost me quite early in the piece was
>"A side effect. You're asking me to believe in sentient weights."
Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.
It's a poor adaptation of the line in the original story (https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...)
The original:
>That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat.
I think it was adapted by an LLM which didn't quite get the meaning.
I noticed that jump too, but I also don’t think the interlocutor is supposed to be 100% logical and rational. The aliens in the original weren’t either.
I didn't read it as coming to the same conclusion as the original, because the meat story presupposes that we who are meat already know that the aliens are wrong. (Maybe that's a humanist reading of the original, but okay). I didn't read this one as trying to make a case that we are fools for assuming that matrix multiplication can't be intelligent... I think its point was that it can't be intelligent, and that people trying to judge it the way mechanized aliens would judge meat creatures just makes them sound ridiculous.
Don't take this as a criticism, but I think overwhelmingly people took it the other way. The fact that the author admits at the end that the story was written with the assistance of "weights" is a tell, to me. I just have to assume the author's genuine position (which I believe to be, we don't know that LLMs aren't conscious or that they could never be conscious) is so absurd to you that the thing comes across as satire. I find myself in that same position sometimes.