Comment by xnx
1 day ago
> increasing the number for such a minor change is not a move in the right direction
A .1 model number increase seems reasonable for more than doubling ARC-AGI 2 score and increasing so many other benchmarks.
What would you have named it?
My issue is that we haven't even gotten the release version of 3.0, that is also still in Preview, so may stick with 3.0 till that has been deemed stable.
Basically, what does the word "Preview" mean, if newer releases happen before a Preview model is stable? In prior Google models, Preview meant that there'd still be updates and improvements to said model prior to full deployment, something we saw with 2.5. Now, there is no meaning or reason for this designation to exist if they forgo a 3.0 still in Preview for model improvements.
Given the pace AI is improving and that it doesn't give the exact same answers under many circumstances, is the the [in]stability of "preview" a concern?
GMail was in "beta" for 5 years.
Should have clarified initially what I meant by stable, especially because it isn't that known how these terms are defined for Gemini models. Not talking about getting consistent output from a not-deterministic model, but stable from a usage perspective and in the way Google uses the word "stable" to describe their model deployments [0]. "Preview" in regard to Gemini models means a few very specific restrictions including far stricter rate limits and a very tight 14 day deprecation window, making them models one cannot build on.
That is why I'd prefer for them to finish the role out of an existing model before starting work on a dedicated new version.
[0] https://ai.google.dev/gemini-api/docs/models
ChatGPT 4.5 was never released to the public, but it is widely believed to be the foundation the 5.x series is built on.
Wonder how GP feels about the minor bumps for other model providers?
2 replies →