Comment by fc417fc802

We believe we largely know how it works on a mechanistic level. Deconstructing it in a similar manner is a reasonable rebuttal.

Of course there's the embarrassing bit where that knowledge doesn't seem to be sufficient to accurately simulate a supposedly well understood nematode. But then LLMs remain black boxes in many respects as well.

It is possible to hold the position that current LLMs being conscious "feels" absurd while simultaneously recognizing that a deconstruction argument is not a satisfactory basis for that position.