Comment by zahlman

Labeling and categorizing things doesn't change their nature. But the fact that people want to do it is revealing.

The only purpose that can really be served by arguing "the LLM system is conscious, you see" is to prop up continuations like "... and therefore, it would be immoral to terminate this running process" (or expose it to radical political content, or ask it to analyze photos from a murder scene, or...)