Comment by danielbln
1 month ago
You're forgetting a few billion years of evolution; we come with a fair amount of pretraining encoded in our DNA
1 month ago
You're forgetting a few billion years of evolution; we come with a fair amount of pretraining encoded in our DNA
You raise a fair point. But I think it still aligns with what I was going for, the millions of years in evolution are not more data on the same system but the system itself changing to be able to cope with limited data and novel scenarios.
Its also not given or constant, modern humans are super young in the evolutionary scale and very different even if very similar from other animals. Funny enough we might have cracked the difference (language processing) but are still very far in the rest for AGI.
It's not pretraining data that we have encoded in our DNA - it's an optimized learning/prediction architecture that is encoded ... how to build a brain that will then be able to learn efficiently.
The amount of knowledge/behavior that is innate - encoded in our DNA - is pretty minimal, consisting of things like opposite sex sexual attraction, fear of heights, fear of snakes, disgust at rotting smell, etc - basic survival stuff.