Comment by pingou

> Redefining consciousness as "has emergent behaviour" is another take that would have rightfully gotten one ridiculed 5 years ago

And what does the fact that it now doesn't show?

>the emergent behaviour is exactly in line with what you'd expect from their relatively simple programming and has zero indications of the complexity of human biological programming.

Well, five years ago, many doubted that they would achieve this much, so it is easy to say now that it is exactly in line with what we expect. And again, the fact that it is different from biological programming proves nothing. It seems much harder to prove that they aren't conscious than to simply say, "I don't know", let alone to claim that they will not become conscious if scaling continues, or if we give them goals, a synthetic sense of worth or self-preservation, or something else.

> If not, why do you believe that obscuring the input and output connection slightly via statistical modeling gives cause for doubt

My hunch is that it is indeed impossible to prove that they are conscious based on their output alone, any more than I can prove that you are conscious just by listening to you. Yet, I believe there is value in listening to what they have to say, perhaps they can come up with a convincing argument.