Comment by selcuka

They accept more tokens these days, but they are still more accurate with a shorter context [1].

[1] https://arxiv.org/abs/2307.03172