Comment by throwuxiytayq
17 hours ago
I don’t mean to advertise OpenAI - let’s make it clear, fuck OpenAI - but I’ve never seen a degradation like that in Codex. All models have always seemed completely stable over their release lifetime. Meanwhile, I rolled back my attempts at using open weight models because providers start throwing “too many requests” errors after just a few requests and the pricing is roughly 10x worse for same model quality, except the inference is much slower. Getting your weights silently downgraded sounds like fun.
OpenAI does the same, the day Astra released I asked it to make a game, it made a really detailed beautiful blender model and some gameplay elements.
A week later, same prompt, really low poly blender model. Either they reduced token usage per person, or they quantized the model, idk, but it really doesn’t work as good as day of release anymore
I keep hearing these anecdotes, but never see it in practice, nor in benchmarks. Do you think it's possible that there exists someone who asked Astra to make a game and they got an ugly one on day 1 and a nicer one later? And don't you think blender modeling is a bit of a "svg of a pelican" problem? It's not what the type of task the model is trained to be doing, and I wouldn't really expect the results to be particularly good or reproducible.