Comment by ACCount37

Does it matter? Evolution is the brain's very own "pre-training". Hundreds of millions of years of priors hardwired.

We can do that for AIs too - pre-train on pure low Kolmogorov complexity synthetics. The AI then "knows things" before it sees any real data. Advantageous sometimes. Hard to pick compute efficient synthetics though.