Comment by akersten

> Interesting, I don’t see this very often with the latest models.

Strange response. I agree with the parent comment here, plan mode lets me ensure that I have specified everything correctly before it gets built which is far too late. I don't see how an improved models even matter to this workflow. Is Fable going to read my mind?

The model is irrelevant, plan mode helps us ensure the model will actually do what we have in mind. The best model in the world can't work around misinterpretation because of bad specs. I'm not sure why this is even a discussion, isn't it obvious to anyone actually using LLMs?

This is an out of touch with reality thing to say. Remember: not everyone has access to unlimited usage/money to spend tokens or the most expensive models with the higher thinking settings, exceptions does not make the rules.

I don't think me giving the model bad instructions is something a smarter model can solve. I use plan mode constantly (with opus/fable), and at least once a day I'll say something too vague or just dumb and it will sketch out the "wrong" solution in it's plan.

Which is fine because it just put together a plan and didn't spend 10 minutes rearchitecting everything.

I haven’t tested with Claude specifically in a while, but I see this a lot on larger features.

It tends to be small decisions way down the stack that bubble up, or an incoherent data model that can’t handle what you’re asking for cleanly.

Eg I was messing with a state tracker the other day. The state tracker assumes a container is either currently running, or fully removed from disk.

The LLM chose to remove the state file when the container is stopped and then to remove it after, which leaks container storage.

The LLM is kind of stuck though, because every option other than “rewrite the data model” has negative outcomes and it probably violates user expectations to launch a massive rewrite there.

I have primarily used Fable 5, Fable 5.1, and now Opus 5.5. I never use shift-tab to enter plan mode personally. My general workflow is chatting with the machine to work out fiddly bits, then /plan to get everything in one document, and reviewing the plan.

/plan is still useful, I still need to review what it's going to do and still make revisions. But there's two phases: hammer out key design decisions then write and amend the document.

It happens with Opus when I leave too much up to interpretation and the AI doesn't do what I had envisioned but didn't specify. Like sure, what it did may be a technically correct solution but it's not the correct solution that allows for further development of my idea. I'm not sure how others do their projects but I start small with proof of concepts and develop in layers until the project does what I want. I use plan mode first to layout everything I can think of that I ultimately want and describe features in the best detail I can manage. I work with Claude to figure out the best framework or find whatever existing projects can serve as a starting point. The first milestone is the proof of concept, take the framework/existing project and build something that does the bare minimum of what I need in the way I want it done then build a test suite to make sure it works. Once that's proven out, we start adding more features (both mine and the ones Claude has suggested) and adding/revising tests along the way. For small things, I won't bother with a plan since I generally already know what I want or any ambiguities can be solved in a single response. But for larger things, I try to take a waterfall approach with well defined milestones.

If I knew exactly how I was going to build something, I would have built it myself. But since there's some ambiguity in the portions of the project I'm less familiar with, I rely on the plan to not only help me understand the decisions Claude has made for me but to keep Claude constrained to the decisions I've made. It's very frustrating to waste tokens on having to refactor something because

I use fable 5.1 (tried opus but it lied to me 3 times in quick succession and ignored me in another)

This is weird to ask because I feel like of course the model isn't omniscient? Isn't the whole point of iterating on a plan to assess impact, risk, know your (the user) variables, user impact, product impact, etc for making a change? I cannot count the times even in the past few months where I start a conversation with my C suite because their desired outcome would have a potential negative impact elsewhere for other products or users.

Is this just not something that comes up at Anthropic?

We got Opus 5.5 just a few days ago so haven’t had time to learn its ways, and Fable was/is too expensive for many people including myself. Insiders have months to learn the ins and outs of new models.

It would be nice if with new model releases claude code also gave a bit of a model 101 that tells you evolving ways of prompting it that the insiders have picked up. I know there’s the prompting guide in the claude docs, but this is often very broad and most people don’t know about it.

Hey, well first off, congrats on making the greatest product ever probably.

I absolutely see fable and opus 5.5 misunderstanding intent, but that just seems to be a feature of necessarily underspecifying in a written prompt. Just today, I gave opus 5.5 a simple task to spin up a new environment for work. It read the ticket, which was decently specified and knowing the codebase as well as "Ghasp... reading the code" I had to correct it about 5 times to do it in a way that I would have expected it to. Getting the pipelines right, environment variables, and configs. It was all relatively straight forward imo. Then I had to prompt it to clean up its corrections, because it left a workflow variable in the github action that some intermediate step required but the final solution didn't. I definitely would not have caught that if I didn't read the output. Idk, there seems to be a natural limit as to how much it can infer and I have no idea how to fix it. I did write about it [here](https://javiergonzalez.io/blog/the-assumption-problem/) though.

Why is “are you using the latest model?” the automatic response to any even mildly critical of LLM coding?

OP wasn’t even actually critical of LLMs, they were just saying that plan mode was helpful to stop the model from making incorrect assumptions when you want you don’t specify everything you should.

For me, every day, although it fails far less often at this than Opus 5 did, which might as well have been a gremlin. It was a really poor technical writer too, leaving slop reminders to itself in all prose, including comments. So even if 5.5 fails less right this second, I have trouble believing that it all will not similarly bite me next week.

Summaries that don't tell me when it's changed direction in a timely fashion, but I am only told way later, when I have to undo. Really bad judgement calls regarding where to fix bugs, changes in implementation decisions, taking action when I am asking a question directly, not passive aggressively asking for action... when 5, 3 days ago, was proven to be untrustworthy, switching to very little supervision sounds like a strange thing for a customer to do.

How could a newer model be better at making up information? Do you understand basic information theory? Or maybe what you're saying is you don't actually have any ideas and are happy to do what everyone else has done before you so it makes no difference?

Hi! Taking this moment to gripe; forgive me...

5.5 and 5.1 have major Rain Man (savant) syndrome. Excellent at many hyper-technical things, absofuckinglutely boneheaded at anything that a human (or an earlier model) would understand - like how to write copy, what a human would expect in a given situation, various types of norms...

it's infuriating because it's a sophies choice - dumber model but better human understanding, or better technical model that you have to explain things to over and over like a toddler.

This is the process I generally use too. Small plans you can just ask for, and big plans you work through building a plan.md file before you build it.

That also makes it easier to adversarially review the plan (I have Fable write the plan, then review it with Astra and another Fable instance).

I guess that You dont need plan mode any more is ambiguous. I took it to mean you dont need to plan because the models are so good at infering intent. You are taking it as "you dont need a dedicated modality to create a plan".