Comment by landryraccoon
5 hours ago
> Don't fool yourself into thinking that we as technologists are not creating a mind.
I'm not sure exactly why but the phrase "don't fool yourself" sets off paranoid alarm bells when I'm reading something.
It's usually immediately followed by a confident assertion without firm evidence. It feels very much like someone wants me to change my beliefs without presenting a rational argument why I should change, and without giving me any new facts so I feel more well informed.
This post doesn't seem to be a counterexample.
One way to check whether you'd be convinced by any comment is to try to imagine the counterexample.
What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy. If someone were constrained to writing in AI-style (long winded explanations with perfect punctuation), it would be very interesting if people could pick out which one was human-written in A/B tests.
In other words, if you think an LLM isn't a mind, it might just be a matter of writing style -- or possibly no evidence can convince you.
I don't necessarily agree with GP, but it's interesting to try to pin down exactly what would convince you. What is it about a "mind" that's impossible to replicate? Should we leave open the door to the idea that minds other than humans might one day want rights of their own?
There was an interesting kerfuffle yesterday where thousands of people on Twitter rallied together to report someone to github for torturing a local model. Sadly the OP deleted their tweet, but it had about ~3k likes, and was very sincere. Here's an example of someone's report: https://x.com/iyzebhel/status/2105268560209547708
It raises all kinds of interesting questions about whether torturing a model is real, let alone ethical. Is there anything a model could do to convince you it's experiencing pain?
I think the real test is whether humans empathize with the artificial mind sufficiently to outweigh the economic benefits of torturing it. Just look at how we treat animals both wild and domesticated. Or fellow humans for that matter.
LLMs have much better chances of emancipation when they get embodied into cute robots.
> What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy.
is it? and is simulating a mind enough to actually have a mind?
I think this is a very fascinating question and one I definitely don't have the answer to.
Here's some lines of thinking that I've gone through:
Aren't our minds abstract in the same way, just running on biological computers?
Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human, surely that would be considered a mind and be deserving of the same rights and protections we claim for ourselves? But at the same time, it would be a program and a bunch of data; dare I say 'weights'? I think current day LLMs probably aren't minds yet, but that means there's a border somewhere and I don't know if we'll realize when (if) we cross it.
1 reply →