A discovery by a giant is in some sense a new base vector in the space of discoveries. The interesting question is if a statistical machine can only perform a linear combination in the space of discoveries, or if a statistical machine can discover a new base vector in the space of discoveries.. whatever that is.
For sure we know modern LLMs and AIs are not constrained by anything particularly close to simple linear combinations, by virtue of their depth and non-linear activation functions.
But yes, it is not yet clear to what degree there can be (non-linear) extrapolation in the learned semantic spaces here.
Probably not homo sapiens.. other hominids older than us developed a lot of technology
A discovery by a giant is in some sense a new base vector in the space of discoveries. The interesting question is if a statistical machine can only perform a linear combination in the space of discoveries, or if a statistical machine can discover a new base vector in the space of discoveries.. whatever that is.
For sure we know modern LLMs and AIs are not constrained by anything particularly close to simple linear combinations, by virtue of their depth and non-linear activation functions.
But yes, it is not yet clear to what degree there can be (non-linear) extrapolation in the learned semantic spaces here.
Pythagoras is the turtle.
Pythagoras learned from Egyptians that have been largely erased by euro/western narratives of superiority.