Comment by throwanem

Oh, I see. That's too broad a claim in my view, but I would agree we can't be certain without a general solution of the 'hard problem' - no more about LLMs than about humans; in the general case, we can't prove ourselves conscious either, which is the sort of thing that tends to drive consciousness researchers gradually but definitely up a wall over time. (But we've discussed Hoel already. To his credit, he's always been very open about his reasons for having departed academia.)

It sounds to me as though you might seek to get at a concern less mechanistic than moral or ethical, and my advice in such case would be to address that concern directly. If you try to tell me that because LLMs produce speech they must be presumptively treated as if able to suffer, I'm going to tell you that's nonsense, as indeed I have just finished doing. If you tell me instead that they must be so treated because we have no way to be sure they don't suffer, I'll see no cause to argue. But I appreciate that making a compassionate argument for its own sake isn't a very good way to convince anyone around here.