Comment by minimaxir

Only if you go above 200k, which is a) standard with other model providers and b) intuitive as compute scales with context length.