Comment by varispeed
6 hours ago
The comparisons with models like GPT-5.6 or Opus are pointless because they vary substantially. They are often nerfed shortly after release and then performance is not consistent. One day model will be doing relatively fine, another it will be dumb as markov generator. In that regard self hosted models are better, because you know what is running your requests.
there are some efforts to track it: https://github.com/ninjahawk/livenerf
This is thought to be conspiracy thinking but I don’t think it is