Comment by Alifatisk
3 days ago
Does it reset at every turn? From my experience in Codex for example, Luna (Max) fills the 256k token window relatively quick. The only thing lowering the context window again is the compaction.
3 days ago
Does it reset at every turn? From my experience in Codex for example, Luna (Max) fills the 256k token window relatively quick. The only thing lowering the context window again is the compaction.
I mean the thinking does not bloat the context window because it gets dropped at the next request.
It doesn't. It's called preserved reasoning and every recent reasoning model does it
Sorry, I've realised I was only partially correct.
Gemini[0] for example passes along a snapshot of the reasoning state but it's not the equivalent to keeping all the reasoning tokens in the context.
[0] https://ai.google.dev/gemini-api/docs/thinking#signatures
Edit: Apparently it does take the same space in the LLM latent space so I was wrong.
1 reply →