Comment by zoogeny

I think this is probably accurate and what remains to be seen is how "compressible" the larger models are.

The fact that we can compress a GPT-3 sized model into an o1 competitor is only the beginning. Maybe there is even more juice to squeeze there?

But even more, how much performance will we get out of o3 sized models? That is what is exciting since they are already performing near Phd levels on most evals.