We believe we largely know how it works on a mechanistic level. Deconstructing it in a similar manner is a reasonable rebuttal.
Of course there's the embarrassing bit where that knowledge doesn't seem to be sufficient to accurately simulate a supposedly well understood nematode. But then LLMs remain black boxes in many respects as well.
It is possible to hold the position that current LLMs being conscious "feels" absurd while simultaneously recognizing that a deconstruction argument is not a satisfactory basis for that position.
We believe we largely know how it works on a mechanistic level. Deconstructing it in a similar manner is a reasonable rebuttal.
Of course there's the embarrassing bit where that knowledge doesn't seem to be sufficient to accurately simulate a supposedly well understood nematode. But then LLMs remain black boxes in many respects as well.
It is possible to hold the position that current LLMs being conscious "feels" absurd while simultaneously recognizing that a deconstruction argument is not a satisfactory basis for that position.
The only reason we know a brain can produce consciousness is because it produces ours
Externally, a brain and an LLM are “just” their constituent interactions.