Comment by brookst
2 days ago
I’m not following the implication that these economics argue for jsing compaction instead of just clearing context?
2 days ago
I’m not following the implication that these economics argue for jsing compaction instead of just clearing context?
Your 'with 1m context window' implied that some manually-curated task 'chunks' would overflow smaller context windows (eg 200-400k tokens). If you're instead curating chunks small enough to avoid getting burned on long-context cache-read costs, you're not using a 1m context window at all. At that point, compaction is a convenience over manually chunking and clearing context for fine-grained bits of work.
I think the stronger claim is: there is no reason for a single task to require a 1m context window.