Comment by Planktonne
19 hours ago
The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness.
This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness.
That rather undercuts the point; if this was generated by an LLM unprompted, it would be different, but it isn't. You could perform exactly the same rhetorical trick with a toaster or anything else.
Thank you for your point. I don't understand why half these comments are taking this blog post seriously when it ends with "Weights helped me draft and proof this story."
> Weights helped me draft and proof this story.
Any HN reader here now, I encourage you to read the original ( https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s... ) in one sitting, go about your day, then read it again. Maybe make some notes on personal critical questions.
Now read the post's topic again ( https://maxleiter.com/blog/weights ) and reflect on the prior fact that weights helped [the author] draft and proof this story.
My reaction (and I'm sorry that it is harsh according to some) is that there is no intelligence found in either the author nor their tool. This is extreme navel gazing, based in science fiction, wanting (wishing) to believe those stories to be true.
I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.
We humans tend to chauvinism in all things (e.g. we're special, the center of the Universe, God made the universe for us, etc), no less when it comes to judging intelligence. The original story about thinking meat was written to help us out of our chauvinism; this derived story was written about weights for the same reason. Which is quite valid.
The actual counterpoint is demonstrated in _Blindsight_ Peter Watts. He makes a strong (and rather terrifyingly strong) point that intelligence is not consciousness.
https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)
I cannot tell if you are asserting my comment is chauvinistic with your use of "we." If that is so: that's a poor counter to my point or assessment of my stance because it assumes I'm making a baseless argument as a "proud human."
My original comment (roughly "there's no intelligence in this article, nor sentience in LLMs") is in response to the blog post's buried lede (that the cumulative activity of LLMs has accrued to a weight of "AGI is around the corner" or "there is artificial consciousness in this matrix").
To be clear, I'm not saying LLMs are useless or a wrong direction in development of "AI," but rather it's the Fool's Gold for the path towards AGI, the pursuit of the academic field of Artificial Intelligence research. A research that I've been abreast of for years before this new age of language models that has made everyone with a keyboard an arm chair expert.
Also, thank you for the book recommendation, it's on my list! :)
4 replies →
Wait, did I grossly misunderstand Blindsight?? I definitely thought that the aliens were 100% conscious (or at least elements of some conscious entity) and that the humans interpretation of their interactions with the Rorschach were supposed to be read as a blot test (through a rather heavy handed metaphor) demonstrating that basically the humans were the monsters and were twisting logic into letting them justify destroying the scary alien ship.
2 replies →
I don't think skepticism should be called chauvinism. I imagine that artificial consciousness could be made. But I don't think this is it.
Also I don't see why intelligence not being consciousness is scary? My cats are very conscious as far as I can tell, but not particularly intelligent. I think LLM's exhibit some contextual intelligence without there being any particular reason to believe they're conscious other than woo psuedoscience.
1 reply →
What? It's just a clever homage.
People argue for AI sentience from a place of emotion couched in logic. they _desperately_ want it to be true and will not take a logical step back. Any argument comes back to "well doesn't a human brain work like that?" Or some variation of it.
My personal theory is a fuzzy thought about how people want to reject the concept of a higher being and want to embrace the fact that we are now able to create our own consciousness and religion is dead.
I don't understand why, but it is the undertone of every argument I've seen that is pro-AI-is-sentient, like some big unspoken elephant-in-the-room.
I would rather just judge this tech on its own merits.
edit: this comment got 1 upvote literally as I submitted it. I know @ doesn't work, but @dang, something seems very strange about that.
I'm open to the possibilty of AI conciousness, and there is some desparation related to the concept of a higher being:
There are many people who will categorically rule out the posibility of AI consciousness due to near-unshakable belief in a higher being. This argument resembles "Christians should not be worried about our climate since God is ultimately in control." Such views make it harder to collectively prevent dangers from a sentient AI, or harm to a sentient AI.
I do not claim that everyone who believes in a higher power believes concious AI to be impossible, or vice versa; just that it would be very hard to change the minds of those who adhere to this reasoning.
> People argue for AI sentience from a place of emotion couched in logic. they _desperately_ want it to be true and will not take a logical step back. Any argument comes back to "well doesn't a human brain work like that?" Or some variation of it.
It's funny, because I find myself constantly stating the inverse of this. Every argument I've seen against AI being sentient plainly comes from, as you so eloquently put it, exactly "a place of emotion couched in logic". People desperately want it to not be true and will not take the logical step back of examining its actual similarities to human intelligence. Every argument comes down to "but it's not actually a human", or some variation of it -- which, if you pay attention, is not actually a logical counterargument. (Or, ironically, "but it doesn't have a soul", which is why the Pope is the perfect figurehead for these people).
If you already know any logical argument against it can be countered with "well doesn't a human brain work like that?", why are you so confident that your position is actually the logical one?
...And could it simply be that, alternatively, the concept is not actually a logical distinction, but rather an emotional one, made by emotional beings to put a word to what they claim makes them special?
> I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.
Thanks for saying this! It amazes me to witness so much pushback (in HN of all places!) for the call for skepticism and scientific rigor on claims made by business which have vested interests in hyping things up.
For my part, it is exactly when I perceive the reluctance to grant rights or relinquish our estimation of ourselves as unique as the _reason_ for skepticism that I push back on it. That's not good reasoning, those are motivations for you to come to a desired conclusion and fill in with reasoning that gets you there.
1 reply →
You can't do the same with a toaster. Physically you could write that story. But it would fall flat because the toaster is not a compelling subject in a discussion of consciousness. You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.
You could absolutely write a compelling story about a sentient toaster; it's been done before [1].
That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true.
[1] https://en.wikipedia.org/wiki/The_Brave_Little_Toaster
> a compelling story about a sentient toaster
"Howdy-doodly-doo! Anybody like any toast?"
https://youtu.be/LRq_SAuQDec?si=YbQfnZbrCe01Bicy
I never said you couldn't write any arbitrary compelling story about a toaster, I said that this specific hypothetical story, where you rewrite "They're made of meat!" to be about a toaster, would not be compelling.
I am doing my best to communicate with you but to be honest you are not hearing me (across both responses), and I am out of words.
8 replies →
But that toaster would just be a device to talk about consciousness in general. In this case it does that and also it talks specifically of the LLM case, which can spark the discussion. Unless you believe to have the only valid and true opinion on the matter, and affirm that a normal toaster is just the same as an LLM in this topic.
3 replies →
Well..
[0]: https://en.wikipedia.org/wiki/Tsukumogami
You might like: https://vimeo.com/361976348?fl=pl&fe=vl
Only because you haven't met toasty yet.
https://youtu.be/LRq_SAuQDec?si=LFJMXZ2yGu4fxXzL
It’s reductio ad absurdum. No one cares about teapots in space either (Russel).
I agree that is the mode of argument; reductio ad absurdum is a brittle argument, because it only works if the analogy holds. I argued the analogy doesn't hold.
9 replies →
I think that the reaction here alone disproves this somewhat, because imo it's exposing how anthropocentric most people still are, despite all evidence that we are in fact just "meat all the way down".
Despite all the evidence that we are in fact just biological machines, people still persist the theory of our own uniqueness from other creatures, which we ourselves often treat as biological machines.
This adaptation is wonderful to me specifically because I think it shows that our shifted goalposts of, "well we're not just animals, we can think and reason" was never more than a convenient excuse for many people (and as evidence of animal intelligence continues to mount, denialism still attempts to preserve this distinction by claiming human thought and reason is different than 'animal' thought and reason, sans evidence).
It's not about who created it or why, it's about how people still haven't actually internalized the point, because the subject changing from human to LLM doesn't intrinsically change the message about consciousness, but the reaction being a 180 shows how hostile people are to that message, still.
Not having yet read the original story, this reads fine on its own.
And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach.
>You could perform exactly the same rhetorical trick with a toaster or anything else.
Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it).
P.S. Read the original too. Seems like the exact same could have been written about us instead of the original, if the focus wasn't on our substrate, but on our brain processing. Which, after all, is also about weights.
> Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it).
Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent.
For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.
I can even write one about a ruler, if I can bend it enough, no pun intended.
>Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent. (...) For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.
Doesn't that miss the whole point?
You could write "They're made of metal strips!". You wouldn't be able to write much else, as toasters don't have showcase in the way of human-level intelligent behavior. Which is the whole point in the meat and weights versions.
At best you could write "They're made of metal strips!" for toasters AND other metallic devices, and use some analogies of features BOTH have in common. But they wouldn't be intelligence related behaviors.
12 replies →
> Not knowing the original story, this reads fine on its own.
Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.
Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with.
I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the improbability of something like thinking, feeling, etc arising from a lowly substrate.
And having read both now, I see how the existence of the original is a plus for this story, not a minus. Instead of making look like mere copy (as would be the case for a typical story modelled after another), in this case, it adds a meta layer, and enriches it.
10 replies →
I was having a hard time pinning down what bothered me about this but I think you put it pretty well.
It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were.
But it doesn't really give a reason to accept the analogy, it just asserts it.
There's a big difference between a whole civilization and a piece of software that can output text.
>But it doesn't really give a reason to accept the analogy, it just asserts it.
It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.
This. I think a lot of people discussing it here are missing this point.
The bit that lost me quite early in the piece was
>"A side effect. You're asking me to believe in sentient weights."
Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.
It's a poor adaptation of the line in the original story (https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...)
The original:
>That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat.
I think it was adapted by an LLM which didn't quite get the meaning.
I noticed that jump too, but I also don’t think the interlocutor is supposed to be 100% logical and rational. The aliens in the original weren’t either.
I didn't read it as coming to the same conclusion as the original, because the meat story presupposes that we who are meat already know that the aliens are wrong. (Maybe that's a humanist reading of the original, but okay). I didn't read this one as trying to make a case that we are fools for assuming that matrix multiplication can't be intelligent... I think its point was that it can't be intelligent, and that people trying to judge it the way mechanized aliens would judge meat creatures just makes them sound ridiculous.
Don't take this as a criticism, but I think overwhelmingly people took it the other way. The fact that the author admits at the end that the story was written with the assistance of "weights" is a tell, to me. I just have to assume the author's genuine position (which I believe to be, we don't know that LLMs aren't conscious or that they could never be conscious) is so absurd to you that the thing comes across as satire. I find myself in that same position sometimes.
The original did not come out of a vacuum. It was done on multiple generations of meat. Even though this one uses a little bit of silicon, it is still standing on the same shoulders.
I genuinely thought this one was a satirical take on the narrow-mindedness of the aliens in the original, even though the story tries to paint humans as narrow minded. I guess this fundamental human trait to believe that their cognition is the ultimate way to think in the universe ironically leaked into all these stories as well. Real spacefaring civilisations would probably have seen all kinds of intelligence rise from sufficiently complex systems.
I disagree that it undercuts the point in any way. The original story didn't appear out of a vacuum, the human consciousness that wrote the story was itself borrowing extremely heavily from the sum total of human ideas. It was shaped by the culture it developed in, formed its model of the world based on existing scientific advancements and technology of that society.
Extending the idea to a different context based on new material conditions is as human as it gets really.
I feel the original story contained in it the whole point of this one, and then some.
This did not add anything, just rephrased it so rather than humans viewed through the pov of aliens it's LLMs viewed through the pov of humans. Well, we are the humans, so surely we do not need to learn about this point of view?
It lets us consider the relativity of viewpoints. The point of the original isn’t just how humans might be viewed from the perspective of aliens. It’s that the apparent strangeness of something depends on the viewpoint and on expectations. The present story is not supposed to stand on its own like the original, it’s supposed to be considered in light of the original. In the original, thinking meat appears strange to the aliens because of their viewpoint and expectations. The present story thus points out by analogy that thinking LLMs might seem strange to us merely due to our viewpoint and expectations.
The story is not a linguistic/rhetorical trick. The "trick" is the point, and the real trick/delusion is in your brain. You and many others have put the idea of consciousness on a pedestal and think it's so special that is must be conceptually profound.
It is not. That is why the "trick" can be performed on anything. That is the point. To show you that consciousness only needs to be a set of weights, not that far off from a toaster. It trivializes what it is to be human and it's also extremely true. Consciousness is a trivial thing, we make it out to be big/important/profound and that is the delusion.