Comment by intalentive
2 years ago
Claude Shannon showed you could use n-grams to represent language. Then Chomsky invented automata / formal languages theory to show why it couldn’t work.
IIRC transformers are at the very bottom of the Chomsky hierarchy, and yet they are clearly able to master English grammar.
What gives?
Have they really mastered language or are they mimicking it.
Do Submarines swim like fish?
Have planes mastered flight or are they mimicking it ? Can you mimick flight ? Is that a distinction that has any meaning ?
Yes. Planes don't fly the same way that birds do.
They are go up in the air like birds but what they are doing to get up & be up in the sky is very different.
Just like LLMs, planes are way less energy efficient than birds. Just like how Human brains are way more efficient than Convents & LLMs
10 replies →
If it quacks like a duck, walks like a duck, is it a duck?
We haven't made machines that replace animals. A submarine can't do quick precision movement to pick up tiny objects like a fish, a plane can't do quick precision movement to fly through a forest and land gracefully on a branch like a bird.
The current machine intelligence is still clumsy like that, it can solve some problems but it is still far from the real thing.