Comment by Tenoke
14 hours ago
Claude seems to forget what you tell it in very long work sessions (things that take weeks to develop), no matter how many times you tell it which part is extra important. I dont use goal (I guess I should), but presumably it makes it actually remember the most important instruction. I believe this here is about shorter sessions where the issue doesn't crop up as much.
I put context size in my Claude Code status bar and ... it does get "tired" when it's at 50%-60% context. I figured this out when I'm like "now modify the testbed to run this and add a test to exercise blah blah blah" and it was just like "That would be a substantial infra lift." In my head I say "I am paying a substantial amount for you to do this!" but just type /compact and re-prompt and my "substantial infra lift" is done without complaint.
I guess everyone needs a nap after a long day of conversing and writing code. So like us!
/compact is prone to error and I wouldn't recommend it in the middle of work. But when you are switching to a related but not completely new task, it helps. ("Now write the integration tests." vs. "on foo.go line 476 that you just wrote, I think there is a deadlock with bar.go line 123 that you just added". It doesn't really need the context to write the tests, it can get that by reading the code. But for iterating on lines of code it just produced, /compact is going to throw away whatever "thought process" led to that code and it's usually not a great thing to do.)
This is one of the advantages of pi. I made a /protect command that protects the message from compaction. I also protect skills automatically.
So for long running tasks I'll do
I am curious as to how this protect works. I am currently working on a pi brain extensions and one of the commands that I will support is:
/brains inspect
that will let you inspect a session and one of the features is the ability to review a compacted message like so:
https://gitsense.com/screenshots/inspect-pi-session-compacte...
https://gitsense.com/screenshots/inspect-pi-session-compacte...
https://gitsense.com/screenshots/inspect-pi-session-compacte...
With /compact in Pi, it creates a message that you can easily review and I am curious as to how '/protect' works.
One of the features that I am working on is to make it easy for agents to retrieve the exact message/event before compaction and I am curious if /protect is a deterministic process or if it is just instructions.
I'm also a fan of Pi - although, doesn't Claude code (and most other harnesses) already have "custom summarization instructions" for compaction/summarization that already allow you to do this? Or have you added on something extra?
They way I read it, this is a workaround for pi, not for Claude.
Edit: removed off topic political spam
4 replies →
Unfortunately I’ve used Claude and GPT models for a long time in a variety of harnesses and I agree with you and I think it’s the compaction.
For some reason, codex compaction is like black magic. I’ve never felt like I can just one one continuous thread with other models, Claude I carefully curate when I compact
Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction.
Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after.
It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude) doing the compaction.
Agree. Another consideration is that input token costs are effectively quadratic with the number of conversation turns. The minute you get into a long conversation you can see the costs shoot up.
You can get far more gas out of even a $20 plan of you’re careful to break things up into relatively small discrete steps, clear context regularly and give the model plenty of information to work with.
My workflow for bigger features is to write out a plan document and then proceed in smaller implementation steps, reviewing as I go. If I find something odd, I ask the agent why and often that leads to discovering a new dimension to the problem, which in turn is an opportunity to adjust the approach.
> With 1m context window there is no reason for a single task to require compaction
Only if money is no object. Cache reads are cheap (10% of uncached input costs) but definitely not free, and cached reads dominate session costs at long context lengths. A prompt at 20k context with $0.01 in cached reads would cost $0.40 in cached reads at 800k context, that quickly adds up for long sessions.
1 reply →
You just haven’t worked on tasks that are complicated enough. Occasionally it took more than 1M tokens just to come up with a plausible plan.
Personally I find using /rewind judiciously is better than using /compact. The latter essentially gives you no control of what details to discard, but the former at least has coarse-grained control.
2 replies →
Most models’ reasoning abilities drops off significantly between the 256K-1M token ranges of the context window. There’s too much stuff to “pay attention to” at once.
1 reply →
This is the way.
One interesting harness thing I saw Cursor do is to give the model access to the entire thread.
Even if it doesn't fit in the context window, the model can search through past turns and sanity check if something doesn't seem to be going right, or be prompted to follow an early message, "when starting on a new item, review the first message for how we should approach this"
Unlike other model/harness pairs, codex+gpt also passes an opaque encrypted artifact speculated to be an embedding representing the conversation back to the successor generation which is “denser” or at least higher fidelity than summarized text.
Pretty much, I stopped using Claude for that reason, I shouldn't have to keep track of the context of the model all the time. And god forbid I don't, the amount of time I've lost to Claude just sending it and then lying is not worth it when 5.5 will just do stuff, compact 2-3 times in the task and still not go insane.
This whole 1 million context window is a lie after 300k Claude degrades to unusable, and compact doesnt help. I've had Codex tabs open for weeks, I have to regularly restart Claude.
That's wild to read considering Codex's compaction to me feels like the model becoming demented. Super aggressive, throws away almost everything.
This is not an issue with Claude, but with Claude Code.
With swival.dev you can use long sessions without ever doing any manual compaction or reset, even with Claude models.
Quality degrades long before you hit compaction, compaction is the "Next gas station, 100 miles" sign but you're already in the middle of nowhere.
You don't need to go down the rabbit hole of crazy workflows, but to avoid slop:
- Break down the work into tasks
- New context. Create a plan for one task.
- New context. Implement the plan.
- New context. /code-review the implementation
- New context. Fix the review findings.
- Repeat for next task.
I do this with Fable 5, and the quality is consistently quite good. If the context goes over 50%, the quality will become crap and you end up with 4 duplicates of the same thing across the codebase. Letting a current session review its own work is like asking a student to grade their own paper.
For the vast majority of models I've worked with, 150k tokens of context seems like some magic spot. If you can stay under it then the session will stay on track and the agent won't start getting forgetful and weird.
Just like with real work, it's valuable to break down big tasks into small tasks that you can knock over in a single session. When a session does start getting too big, you just need to ask the agent to reply with a comprehensive handover report and paste it into a new session.
For my quirky workflows I need 100K even to prep Opus 4.8 or higher to begin good work. Not programming, but a complex synthetic architecture for implementing more attentional autonomy for machines.
i found that at 700k-ish context even fable becomes an idiot, maybe openai's decision to cap codex context at 400k is correct, 400k is really a sweet spot where most part of the context is reliable.
Stop using compaction. Stop treating sessions as something you keep alive for weeks. Set up a proper workflow for handing off work and start handing off well before maximum context is hit. It's less efficient, slower and more costly to constantly be sitting at high context and compaction is just not good.
There is no one-size-fits-all, even for programming.
There isn't, but everything that they said above is absolutely true and people who try to fight it will suffer. There is lots of flexibility within those constraints, but the constraints aren't imaginary.
If you want to use LLMs effectively, everything OP said is true. It has been this way since the beginning. There will be a time when long context and compaction will be effective, but now isn't it.
There's a good way and a bad way to use tools. My suggestions were based on my extensive usage of the tools in question.