Comment by api

Small model becomes worse when irrelevant details are in its context window.

Big surprise.

Small models are a lot worse at meta-evaluating the importance of things in their context. To get them to work well it's a lot more important to keep context focused and compact. Larger models can deal with messy context, though they too will often perform better with focused context.