Comment by kypro

Knowledge is surprisingly transferable.

Obviously it's easy to understand why having strong abilities in maths and coding can help with solving problems in physics. But it's also true of knowledge and mental models more broadly.

For example, say somehow you managed to train a frontier model without any data about elephants. If you did this it would be relatively easy to later train elephants into the weights. You'd basically just need to train it about elephants at a very superficial level. After all, it it already knows what a mammal is. It knows what legs are. It knows what eyes are. It understands how it's digestive system works. All the things that makes the biology of elephants difficult doesn't need to be learnt from scratch.

I assure you people on HN will be surprised at how quickly frontier models will become competent in domains that are harder to RL. They won't need anywhere near as much data as people currently believe.