Comment by kypro
15 hours ago
No offence, but you clearly haven't studied this and are making some wrong assumptions here.
> Neural networks are approximators being fed human intellect.
They're not "approximators", that's a far too simplistic way to think of them.
Neural nets create models and deep layers of abstraction around the data we feed them in the same way your brain creates layers of abstractions to reason about the world. AIs can use these abstractions to come to come up with novel things no human has ever thought.
> Therefore they can only approximate the intelligence of humans
They're not just being fed human data though... Modern AIs are typically trained on huge amounts of synthetic data. This is why AlphaZero got so much better than humans at chess and Go - they're not just trained on human data but they generate their own data and train on that. Similar techniques are being deployed on SOA language models too.
> Even if the llm speaks an alien language, it should be similar to human intellect
Is AlphaZero similar to a human chess player? There's no reason to assume this.
> Neural nets create models and deep layers of abstraction around
We don't have any proof of that, but the approximator thing is proven. The rest is just marketing speak.
Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically
> We don't have any proof of that, but the approximator thing is proven.
Not in the way suggested... It's not approximating human intellect. They approximate the target function, and the target function frontier labs are trying to approximate is ultimately a super intelligence...
> Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically
I'll assume we're talking purely about language models for a moment, but if you assume that everything can be represented linguistically, then in theory there is no upper-bound on what can be learnt with synthetic data.