Comment by gtsop

> I think it is useful and even necessary to ask questions like the following: “if someone engages with this system, and comes to the conclusion that it has ethics, what sort of ethics will they be likely to believe the system has? If they come to the conclusion that it has ‘world views,’ what ‘world views’ are they likely to conclude the system has, even if other people think it’s nonsensical to say it has world views?”

Maybe there is some scientific aspect of interest here that i do not grasp, i would assume it can make sense in some context of psychological study. My point is that if you go that route you accept the premise that "something human-like is there", which, by that person's understanding, will have tremendous consequences. Them seeing you accepting their premise (even for study) amplifies their wrong conclusions, that's all I'm saying.

> Surely this is not quite accurate - the material properties - surface roughness, reflectivity, geometry, etc - all influence the appearance of a perceptible image of a person.

These properties are completely irrelevant to the image of the person. They will reflect a rock, a star, a chair, a goose, a human. Similar is my point of LLM, they reflect what you put in there.

It is like puting vegies in the fridge and then opening it up the next day and saying "Woah! There are vegies in my fridge, just like my farm! My friege is farm-like because vegies come out of it."