This would require a conspiracy since Claude models are hosted by Google, Anthropic and Bedrock. All these companies would have to be working together to secretly hamstring the previous models.
I'm not claiming that they do, but just to play the Devil's advocate:
Anthropic controls the full serving stack, so it can change precision, hardware, kernels and settings whenever it likes. Reducing the KV cache precision, for example, saves a lot of memory (and money). It may even be possible to quantise the model on-the-fly when loading it into memory. They don't necessarily have to upload a whole new model.
Even if they can't do the same on all platforms, we can't deterministically compare them anyway.
What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?
This would require a conspiracy since Claude models are hosted by Google, Anthropic and Bedrock. All these companies would have to be working together to secretly hamstring the previous models.
I'm not claiming that they do, but just to play the Devil's advocate:
Anthropic controls the full serving stack, so it can change precision, hardware, kernels and settings whenever it likes. Reducing the KV cache precision, for example, saves a lot of memory (and money). It may even be possible to quantise the model on-the-fly when loading it into memory. They don't necessarily have to upload a whole new model.
Even if they can't do the same on all platforms, we can't deterministically compare them anyway.
1 reply →