← Back to context

Comment by palata

4 days ago

I would debate the fact that LLMs have "cracked natural language comprehension"...

Not that it's not impressive, but LLMs do not "comprehend", for a start.

6 comments

palata

Reply
frenchwhisker  4 days ago

I see what you’re getting at but it does feel like goalposts are being moved, no? By and large we can ask a computer today a question and it will almost certainly spit back a sensible (!= correct) answer. We can ask what the words mean and ask it to translate it to other languages, and we can have a conversation.

  • palata  4 days ago

    > I see what you’re getting at but it does feel like goalposts are being moved, no?

    I don't think so. I am not setting the "goalpost" here, it was expressed as "LLMs have cracked natural language comprehension".

    I just don't think they have. There are tons of statements I would agree with regarding what LLMs have achieved, but this one is not part of them :-).

    • fc417fc802  4 days ago

      What is the metric to measure comprehension?

      How are we disentangling comprehension of natural language itself from comprehension of the subject matter being discussed via said language sample?

      I think that by most reasonable metrics LLMs can reasonably be said to comprehend natural language itself. However they clearly are deficient in logic and reasoning, as well as comprehension of many of the concepts that the natural language is used to express.