Comment by qsera

6 hours ago

For starters, natural brains have the innate ability to differentiate between things that it knows and things that it have no possibility of knowing...

Modern LLMs are fairly good at that as well.

  • But that is bolted on and is not a core behavior.

    • Does it matter? Evolution is the brain's very own "pre-training". Hundreds of millions of years of priors hardwired.

      We can do that for AIs too - pre-train on pure low Kolmogorov complexity synthetics. The AI then "knows things" before it sees any real data. Advantageous sometimes. Hard to pick compute efficient synthetics though.