mostly cost of hardware going down. as models scale, nvidia produces a new hardware generation that outputs more tokens per watt, but those speed gains get eaten by the fact that the model is bigger ie. more expensive to serve.
Also we have no clue whether Anthropics inference margin is compressing or not and they just want to maintain the price.
Distilling from a teacher (Opus 4.5) and scaling RL more.
So less parameters but "better" weights?
this is called progress
I'm asking technically how progress works. What is actually being improved here
mostly cost of hardware going down. as models scale, nvidia produces a new hardware generation that outputs more tokens per watt, but those speed gains get eaten by the fact that the model is bigger ie. more expensive to serve.
Also we have no clue whether Anthropics inference margin is compressing or not and they just want to maintain the price.
Or, we can bleed out cash for a very long time.