Comment by sothatsit

I have been using Fable 5 extensively, and Opus 5 yesterday and today. I have not noticed any step-change improvement in their judgement in what to keep a memory of or not.

I have actively experimented with this as well. I have a reflect skill that actively prompts the models to modify their memory, and have tried to run sessions actively asking the models to consolidate their memories. Fable is noticeably better at this, but still nowhere near good enough.

Fable will still make mistakes where I give feedback on one piece of code and it will create a memory applying that rule everywhere, completely missing the context for why my advice only applied to that one place. It has also made memories of random details about a service that are very unlikely to ever be relevant again, and for things where we could just read the config if we needed to find that information again anyway. And then it will miss making memories of important architectural concerns.

I think auto-memory suffers a similar problem to comments where newer models write better comments, but their choice over when to write comments, and how long those comments should be, still sucks.