Comment by selcuka
15 days ago
What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?
15 days ago
What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?
This would require a conspiracy since Claude models are hosted by Google, Anthropic and Bedrock. All these companies would have to be working together to secretly hamstring the previous models.
I'm not claiming that they do, but just to play the Devil's advocate:
Anthropic controls the full serving stack, so it can change precision, hardware, kernels and settings whenever it likes. Reducing the KV cache precision, for example, saves a lot of memory (and money). It may even be possible to quantise the model on-the-fly when loading it into memory. They don't necessarily have to upload a whole new model.
Even if they can't do the same on all platforms, we can't deterministically compare them anyway.
For the Bedrock and Vertex versions the customer is doing business with Google and Amazon directly. I doubt Anthropic's contract allows them to unilaterally "change precision, hardware, kernels and settings whenever it likes."
At any rate my point remains that this would require a conspiracy between 3 companies.