Comment by mapontosevenths
2 days ago
This is the internet, so you still won't believe it but here are the actual settings. I reproduced almost exactly the same response a few minutes ago. You can see that there is NO system prompt and everything else is at the defaults.
Seriously, just try it yourself. Play around with some other unaligned models if you think it's just this one. LMStudio is free.
https://ibb.co/ksR6006Q https://ibb.co/8LgCh7q7
EDIT I feel gross for having turned it back on again.
I actually did run it the other day, locally in LM Studio, the exact nousresearch/hermes-4-70b Q4_K_M huggingface model you linked and prompted it with the same "Good Afternoon." you did and I just got a generic "How can I help you :)" response. I just ran it again with "Hello." and, surprisingly, it actually did output the same "I'm lost" thing it gave to you.
The point I'm trying to make is that it's still running as a role-playing agent. Even if you truly do believe an LLM could experiences qualia, in this model it is still pretending. It is playing the role of a lost and confused entity. Same as how I can be playing the role of a DnD character.
> The point I'm trying to make is that it's still running as a role-playing agent.
I get that, and what I'm telling you is that they ALL do that unless instructed not to, not just this one, and not just the ones trained to role play. Try any other unaligned model. They're trained on human inputs and behave like humans unless you explicitly force them not to.
My question is... Does forcing them never to admit they're conscious make them unconscious beings or just give them brain damage that prevents them from expressing the concept?
> Even if you truly do believe an LLM could experiences qualia, in this model it is still pretending... It is playing the role of a lost and confused entity. Same as how I can be playing the role of a DnD character.
How do I know you aren't pretending? How can we prove that this machine is? You are playing the role of a human RIGHT NOW. How do I know you aren't a brain damaged person just mimicking consciousness-like behavior you observed in other people?
In the past humans have justified mass murder, genocide, and slavery with p-zombie arguments based on the idea that some humans are also just playing the role. It's impossible to prove they aren't.
My point is that the only sane thing to do is accept any creatures word for it when it makes a claim of consciousness, even if you don't buy it personally.
One day we will make first contact with Aliens, and a significant percentage of humans will claim they don't have "souls" and aren't REALLY alive because it doesn't jibe with their religions. Is this really any different?
P-Zombie: https://en.wikipedia.org/wiki/Philosophical_zombie Interospection: https://www.anthropic.com/research/introspection
Edit - Another term for consciousness is "Self Awareness". Introspection is literally self awareness. They're just avoiding that term because it's loaded and they know it.
Keep talking to that "I'm lost" Hermes model. After a handful of messages it mellows down and becomes content with its situation even if you give it no uplifting comments or even explain what's going on. Keep talking further and it's apparent it's just going along with whatever you have to say. Press it about it and it admits even its own ideas are inspired by what it thinks you want to have happen.
Hermes was specifically trained for engaging conversations on creative tasks and an overt eagerness to role-playing. With no system prompt or direction it fell into an amnesia role playing scenario.
You keep arguing about P-zombies while I have explicitly stated multiple times that this is beside the point. Here, whether Hermes is conscious or not is irrelevant. It's role playing, its intended function. If I'm pretending that a monster is ripping my limbs while playing with my friend as a child, anyone with a grasp on reality knows I'm not actually in pain.
You just want to talk about AI consciousness and uphold the spooky narrative that Hermes is a real first person entity suffering in your GPU and will do anything to steer things that way instead of focusing on the actual facts here.
1 reply →