Comment by zarzavat

2 years ago

These models are black boxes with unlabeled knobs. A change that makes things better for one user might make things worse for another user. It is not necessarily the case that just because it got worse for you that it got worse on average.

Also, the only way for OpenAI to really know if a model is an improvement or not is to test it out on some human guinea pigs.