Comment by doug_durham
11 hours ago
Prior to 5.2 you couldn’t expect to get good answers to questions prior to March 2024. It was arguing with me that Bruno Mars did not have two hit songs in the last year. It’s clear that in 2025 OpenAI used the old 4.0 base model and tried to supercharge it using RLVR. That had very mixed results.
That just means their pretraining data set was older. You can train as many models as you want on the same data.
I’m sure all these AI labs have extensive data gathering, cleanup, and validation processes for new data they train the model on.
Or at least I hope they don’t just download the current state of the web on the day they need to start training the new model and cross their fingers.