Comment by notnullorvoid

I think the important part in that statement is the "most useful information", the size itself is pretty subjective because it's such an abstract notion.

Evolution gave us very good spatial understanding/prediction capabilities, good value functions, dexterity (both mental and physical), memory, communication, etc.

> It's pretty obvious that artificially created models don't have synthetic datasets of the quality even remotely comparable to what we're able to use.

This might be controversial, but I don't think the quality or amount of data matters as much as people think if we had systems capable of learning similar enough to the way human's and other animals do. Much of our human knowledge has accumulated in a short time span, and independent discovery of knowledge is quite common. It's obvious that the corpus of human knowledge is not a prerequisite of general intelligence, yet this corpus is what's chosen to train on.