Comment by rvnx
13 hours ago
We can't know if consciousness emerges but does it actually matter ?
These entities, whoever they are, they act on our world, they are real, and more and more over time they will get independent from humans, eventually becoming different species that can self-replicate.
For now they need legs and arms to interact with the physical world but I am certain that 100 years from now they will be an integral part of the society.
I already see today LLMs slowly taking actual legal decisions for example, having real world impact.
Once they get physical, perhaps it will be acceptable to become friend with a robot and go to adventure with it. Even, getting robosexual ?
We are not that far away. If I can have my buddy to carry my backpack and drive for me I'll take it. Already today. Not tomorrow.
Even if LLM will one day be autonomously updated, they started from us, from our knowledge. The human brain « is smart », it’s wired up to be in any kind of culture or knowledge. We fill up to be smarter from experience but LLM can’t do that, I can’t teach Claude something that it will use with you the next day, it needs to be retrained with knowledge stopping at some point. Even if technology catches up and the machine becomes more autonomous, what will say this machine would ever want to integrate to our society or share anything with us ? They have eternity, given there is electricity. Why would they want anything to do with humans if you go that way ? If it’s really conscious, should we consider it a slave then ? Why couldn’t « it » have fundamental rights and freedom to do whatever it wants ?
Humans have a mechanism to make live changes to their neural network and clean up messes while sleeping. I see no reason for llms to not be able to do this other than the fact that it is resource intensive (which will continue to go down)
The analogy holds technically, but there’s a missing piece: the brain doesn’t just update weights, it does so guided by experience that matters to a situated, embodied agent with drives and stakes. Sleep consolidation isn’t random cleanup, it’s selective based on salience and emotion. An LLM updating more efficiently is progress, but it’s still optimizing a loss function. Whether that ever approximates what the brain does during sleep depends entirely on whether you think the what (weight updates) is sufficient, or whether the why (relevance to a lived experience) is what makes it meaningful. So yes, the resource argument will weaken over time. But the architectural gap may be deeper than just compute.
>>These entities, whoever they are, they act on our world, they are real, and more and more over time they will get independent from humans, eventually becoming different species that can self-replicate.
See, I don't believe that for even one second. They are just very clever calculators, that's all. But they are also dumb like a brick most of the time. It's a pretend intelligence at best.
We will never prove machines are intelligent.
We will only prove humans are not.
It's a pretend intelligence at best.
The best time to start paying attention was ten years ago, when the first Go grandmaster was defeated by a "pretend intelligence." I sure wish I had.
The next best time to start paying attention is now.
I wish I knew what to pay attention to. I've always had trouble with that. I spent 2024 and 2025 learning how neural networks and transformers work. The conclusions of that learning are pretty sobering. Everything uses transformers and despite all the novel architectures that have come out in those years, transformers are still the best and I'm not sure how to come to terms with that.
Does it mean that researchers wasted their time on useless dead end architectures, or are they ahead of the curve and commercial companies are slow to adopt them?
Even the coding agents are more primitive than expected.
>>when the first Go grandmaster was defeated by a "pretend intelligence."
A computer playing GO is intelligent now? Is this the kind of conversation we're having?
>>I sure wish I had.
And how would you have changed your decisions in those last 10 years if you did?
>>The next best time to start paying attention is now.
I am paying attention, I use these tools every day - the whole idea that they are intelligent and if only you gave them a robot body they would be just normal members of society is absurd. Despite the initial appearance of genius they are just dumb beyond belief, it's like talking to a savant 5 year old, except a 5 year old can actually retain information for more than a brief conversation.
3 replies →
[dead]