Comment by seydor
7 hours ago
> Neural nets create models and deep layers of abstraction around
We don't have any proof of that, but the approximator thing is proven. The rest is just marketing speak.
Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically
> We don't have any proof of that, but the approximator thing is proven.
Not in the way suggested... It's not approximating human intellect. They approximate the target function, and the target function frontier labs are trying to approximate is ultimately a super intelligence...
> Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically
I'll assume we're talking purely about language models for a moment, but if you assume that everything can be represented linguistically, then in theory there is no upper-bound on what can be learnt with synthetic data.
> the target function frontier labs are trying to approximate is ultimately a super intelligence
So an unknown function that is even unintelligible to humans? that sounds a nonstarter