Comment by LordDragonfang

10 days ago

> they don’t understand

I have not seen any instance of this frequently-made assertion which is at all justified. It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence (they clearly can comprehend even complex tasks well enough to execute on them, and if you won't call that "understanding", you're playing word games rather than stating an objective fact).

Likewise, agents can literally come to a greater understanding of a problem through trial and error, and there are plenty of mechanisms to retain that knowledge. If you don't want to call that "learning", you're just making a choice to define it in a way more restrictive than how we use it for humans, and intentionally making communication more difficult.

8 comments

LordDragonfang

Reply
FromTheFirstIn  10 days ago

Agents are always combining the same underlying weights to their inputs, relying on the same maps of semi-semantic space and the relationships between those that it was leaning towards at training time. The fact that it’s successful in making lots of people have an Eliza effect doesn’t make it understand something. It’s simulating understanding based on an enormous corpus of text, much of which is people working through things or sharing an understanding of something. Unless you believe that all intellectual activity is about finding the space between words you shouldn’t believe LLMs have any chance at understanding anything.

mellosouls  10 days ago

It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence

"Understanding" has enough philosophical leeway in its use to allow at least the possibility of sentience as a prerequisite.

This is where the discussion about LLM capabilities becomes genuinely difficult, and dismissing that difficulty as "word games" or "spirituality vs evidence" is not helpful.

  • LordDragonfang  8 days ago

    Considering that "sentience" has enough "philosophical leeway" that it's just as reasonable to assert that LLMs are sentient (and at extremes, that they have been sentient for years) -- especially if we are, as you suggest, supposed to include any philosophically possible definition -- I don't think that's a meaningful rebuttal. If no one can agree on whether it's sentient, it's bad faith to choose a fringe definition that hands off its definition to such a nebulous term.

    In fact, I'd argue that statements about what "is" and "is not" sentient relies on even more spirituality and word games for anything that isn't a terran tetrapod.

    For a meaningful -- "helpful" -- discussion on such things, one has to assume that everyone is choosing a definition which is closer to the median usage and relies on not being totally subjective. Furthermore, given the breadth of options, it should be assumed to be a definition which allows which permits the form of the question to be meaningful, rather than begging the question -- if your definition is tautological enough that non-biological entities can't have understanding, you're just expressing dogma rather than having a discussion.

    Anything else is bad faith, or assuming bad faith on the part of the participants.

    • mellosouls  8 days ago

      it's bad faith to choose a fringe definition that hands off its definition to such a nebulous term.

      I do not think it is at all unreasonable or "fringe" to regard understanding as involving intentionality: ie a directedness of thought toward the object-relations being "grasped". That may not be the only possible conception of understanding but it is a mainstream philosophical idea.

      In fact, I'd argue that statements about what "is" and "is not" sentient relies on even more spirituality and word games for anything that isn't a terran tetrapod.

      Then you seem to be confusing "hard to understand" with "meaningless".

      you're just expressing dogma rather than having a discussion.

      Anything else is bad faith, or assuming bad faith on the part of the participants

      Have a think about that (repeated) tone before responding.

      Fwiw I am a long-time believer in consciousness being fully realisable in machines; I think the jury is still out on LLMs.