Comment by jibalt
2 hours ago
What they said, which is true, is that an LLM's inputs measured in bytes of language is much higher than for humans. Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.
> We happen to feed only distilled data to LLMs though. That's why they appear so efficient.
That is wild handwaving that really explains nothing.
But then, most of the comments on this page are wild handwaving, pontificating like a blind man describing an elephant. Very little engages with or matches the quality of Norvig's arguments.
> Throwing in all the non-language bytes of sensory input received by humans is not valid and thus is not apples to apples. Children learning from watching a dog do zoomies around the room or butterflies in the garden is not learning language.
Have you ever had kids? Either you never had, you forgot, or you are intentionally deceiving us (and perhaps yourself) to support your point. If you never had kids, I can tell you you have very poor imagination about how they learn language.