Comment by vrosas
8 hours ago
The problem is, when the context window grows, Claude tends to forget these kinds of rules. It will then do whatever it wants. I had to outright ban comments in the global claude.md, the local claude.md AND write a hook to catch any that still slipped through.
I think people really need to focus more on working with limited contexts rather than trying to work around it. I really try to keep my sessions as short as possible and it helps a ton with keeping Claude (et al) focused.
Specifically, I like the "canary" trick that people have discussed where you add a small, innocuous rule to your CLAUDE.md like "When responding to me, start every sentence with my name." so that when Claude stops doing this, you know you've used way too much context and need to start a new session.
This or you just repeat the initial prompt every 200k tokens
Which gets you to the point where the whole thing is.. still unreliable. Generative text engines are going to generate. This calls for real enforcement in deterministic pre-edit hooks.
And here is where naive people will say something like "Why do I care if robots shit all over the codebase? Code is for machines, I don't expect to deal with it much now". But really externalized CoT like this confuses machines too, wastes tokens, and eventually wastes exponentially many tokens. Agents tend to think it's more real grounding than prompts are, even for comments-in-code. One bad comment poisons everything, then gets copied around as a ground-truth assumption everywhere. Hooks are more real to them than prompts or comments, and even then if you add enforced limits and tell them to externalize CoT ONLY in scratch task-tracking docs.. they will violate comment-enforcement hooks about 25% of the time. That tells you everything you need to know: even with constant reinforcement, they just really want to break this kind of rule.
1 reply →
Yea, I've started making it write linters to check the code that goes out. Anything that can be deterministically measured, gets added to it once we lock it down.
> The problem is, when the context window grows,
You know the problem; then why not address it? Does Compacting the context not help?
Compacting the conversation almost never helps. It is uniformly worse than starting over with fresh context, or rewinding to a last-known-good state. It only exists because it increases engagement.
This does not match my experience. I use long-running orchestrator sessions. Each orchestrator is in charge of planning, writing kick-off prompts for implementers, answering questions from those implementers, doing code reviews and providing feedback, and answering side questions from me when I have them.
Depending on the initiative I might compact a session a dozen times, sometimes more. It is lossy, and the session certainly tends to forget earlier bits as more compactions happen, but overall it's a much better experience than starting fresh and having to re-explain everything.
The only time I compact is if the session goes wildly off-course and the context gets polluted with off-topic conversations.
Also worth noting: with Claude Code you can provide custom instructions when compacting, and instruct the LLM that is in charge of compacting the session to prioritize the retention of specific bits. It can help a lot.
Compaction is a main cause of this problem.
Compacting context compacts context. So Claude forgets a lot during compaction.
Compacting mostly gets rid of reasoning tokens, and honestly it would be nice of reasoning tokens did not constantly follow every follow up query. Asking even a simple/trivial question can have Claude use thousands of tokens. Compacting is good for getting rid of those.
4 replies →
Hooks is the way.
Intermittent nudges