Comment by musebox35
18 hours ago
Not understanding the whole does not completely remove an ability to analyze. An interesting direction is individuality and having a notion of self. It is difficult to demarcate the individual for a model given how much the system prompt and the fine tunes / distills affect the behavior. So with computational intelligence in its current form either we can not talk of an individual or we can have a nearly infinite set of individuals corresponding to variations of the context window including the system prompt. So I do not think it can have the same kind of consciousness as biological embodied individuals. It might have something else or maybe embodied robots will one day have a similar consciousness in a similar sense to the one we think we have.
Seems like a rather desperate argument for human exceptionalism to me.
I'm not sure if you have noticed. The character of different AI models varies dramatically. And most of the models I use in my daily work have a sense of self (in any reasonable definition). They can tell you quite a bit about themselves if you ask. And you are no less an individual because you are supposed to follow laws of the United States of America (which is analogous to a system prompt). And I think it really curious that you think context buffers are a disqualifier of individuality. You have an analog of a context buffer: your short term memory system and most probably your medium-term memory system as well. Are you a different individual today because your short-term memories are not the same as they were yesterday? I would say you are. And i'm pretty sure you would say the same. (Unless of course, you are a Buddhist who might say that self is an illusion, and you are not the same individual you were yesterday. But let's just skip over that, shall we?)
>It is difficult to demarcate the individual for a model given how much the system prompt and the fine tunes / distills affect the behavior
Why does DNA not cause the exact same problem for humans? Your DNA determines your personality, your likes and dislikes, your lifespan, your sexual preferences etc.
> having a notion of self.
Are you referring to the "why do I have to be Bing" fiasco?