Comment by pka
2 days ago
Recently I watched a documentary on the tanzanian Hadza tribe, one of the last hunter gatherer tribes on Earth. Their language is a distinct click and pop language and they regularly imitate animal calls (monkeys, baboons, birds) when they hunt but also when they communicate with each other, tell stories etc.
I think it's not impossible that words evolved as adaptations of the environmental sounds with which our ancestors lived. The human creativity producing DNA is also a remix of preexisting molecules formed under evolutionary pressure, so the view that it's turtles all the way down, unintuitive as it is, may not be so indefensible after all.
I mean what is your criterion on invention here? On the one hand, each specific word could be seen as a new invention. On the other hand, all languages basically correlate strongly with the environment of their users - it's why LLMs turn out to be universal translators - and pattern-matching is hardly an invention, isn't it?
I don't know, but my current intuition is that invention = more or less preexisting ingredients + some tiny spark of creativity ("connecting the dots") rather than this grand myth of "creating something from nothing". And then the tiny spark could itself be a remix of previous tiny sparks (connecting the dots between dot connections, a bit like maps between n, n+1, n+2, ...-dimensional manifolds), and so on.
> and pattern-matching is hardly an invention
Exactly. That's what I meant with turtles all the way down, until the tiny spark disappears into a dot and it turns out that creativity is just an illusion - akin to how nobody would call the gene (the spark, the pattern matching) formed by evolution (the brain connectome) "creative".
But LLMs aren't turtles all the way down, they stop at vector embedded tokenized words.