Comment by simonw
7 hours ago
That used to matter to me back in the days when the best models still only accepted ~32,000 tokens, but these days even the models that run on my laptop are happy with ~100,000 and the hosted models I use take ~200,000 or more.
They accept more tokens these days, but they are still more accurate with a shorter context [1].
[1] https://arxiv.org/abs/2307.03172
If it's one of many tool calls, I'd assume that less is more.