Comment by elwebmaster
17 hours ago
One very unfortunate curse we are stuck with is LLM providers pulling the rug from under us and swapping models with quantized versions during peak demand or on monthly plans. You never know if that's the reason AI starts writing shitty code or it's just the context size that has grown. On top of that there is the queuing mechanism which makes tasks take longer. Sometimes AI is super fast and sometimes a simple task takes 10, 15 or even 20 minutes of "thinking". You may not realize this if you are just using your same subscription but if you switch between providers you can immediately tell. I advise everyone to invest $5 in openrouter, deepseek and qwencode. Just do an experiment and use same model (deepseek-4.1 flash) asking same prompt on same code, then look at the result. Spoiler: you will see trash coming out of openrouter at 1-5 tokens/second, you will see slightly faster and better quality output from qwencloud and when you get to deepseek API where you are paying per token you will not believe this is the same model. Because it isn't.
I don’t mean to advertise OpenAI - let’s make it clear, fuck OpenAI - but I’ve never seen a degradation like that in Codex. All models have always seemed completely stable over their release lifetime. Meanwhile, I rolled back my attempts at using open weight models because providers start throwing “too many requests” errors after just a few requests and the pricing is roughly 10x worse for same model quality, except the inference is much slower. Getting your weights silently downgraded sounds like fun.
OpenAI does the same, the day Astra released I asked it to make a game, it made a really detailed beautiful blender model and some gameplay elements.
A week later, same prompt, really low poly blender model. Either they reduced token usage per person, or they quantized the model, idk, but it really doesn’t work as good as day of release anymore
I keep hearing these anecdotes, but never see it in practice, nor in benchmarks. Do you think it's possible that there exists someone who asked Astra to make a game and they got an ugly one on day 1 and a nicer one later? And don't you think blender modeling is a bit of a "svg of a pelican" problem? It's not what the type of task the model is trained to be doing, and I wouldn't really expect the results to be particularly good or reproducible.
New conspiracy theory just dropped.