Comment by kloop

I still think this is because, on a new release, it works on some prompts the previous ones did badly at, because new weights do well on a different set of prompts.

Then after a few days you notice the prompts that it does badly on that the old ones did fine with and everyone is convinced there's a regression when it's just a different part of prompt space