Comment by somewhereoutth
18 hours ago
Yes the huge discrete stepwise training spend is critical.
Maybe investors will realise that "the only winning move is not to play".
And so we are left with (as was) frontier models getting more and more out of date as whoever their post bankruptcy custodians are tries to eek pennies on the dollar for inference on their decaying property. Perhaps along with local and/or highly specialized models still feeding on the after-glow of the huge amount of training that was (and is no longer) done.
The next AI winter is going to be deep, savage, and long.
> frontier models getting more and more out of date
Why are they getting out of date? Is it because we have new content from the internet that the older models did not have? Or are we simply trying to increase the size of the training data? In other words not more up-todate in terms of time the content was created vs. wanting to use bigger training-input-sets?
lack of new content from the internet will make them go out of date. Not just facts and figures, but (for example) new programming languages/techniques.