Comment by jesse_dot_id
17 hours ago
The plateau is inevitable because their rapacious training methodologies are only viable when there are no defense in place, but information continues to evolve, which means the models will have to be continuously updated, but will be doing so with less and less freely available data.
> with less and less freely available data
My understanding is that the labs ran out of freely available data to train on a while ago, and now primarily rely on human data vendors such as Surge and Mercor to source their data.
We are only just starting to model the physical world. There will be more training on empirical data.