Comment by a_wild_dandan
1 year ago
Like humans, multi-modal frontier LLMs will ignore "removal" as an impertinent typo, or highlight it. This, like everything else in the comment, is either easily debunked (e.g. try it, read the lit. on LLM extrapolation), or so nebulous and handwavy as to be functionally meaningless. We need an FAQ to redirect "statistical parrot" people to, saving words responding to these worn out LLM misconceptions. Maybe I should make one. :/
THe way current empirical models in ML are evaluated and tested ( benchmark datasets) tell you very little to nothing about cognition and intelligence. Mainly because as you hinted , there doesn't seem to be a convincing and watertight benchmark or model of cognition. LLMs or multi-modal LLMs demonstrating impressive performance on a range of tasks is interesting from certain standpoints.
Human perception of such models is frankly not a reliable measure at all as far as gauging capabilities is concerned. Until there's more progess on the nueroscience/computer science (and an intersection of fields probably) and better understanding of the nature of intelligence, this is likely going to remain an open question.
I didn't know that metaphysics, consciousness, and the physical complexities of my neurology are considered solved problems, though I suppose anything is as long as you handwave the unsolved parts as "functionally meaningless".