Comment by rst
7 days ago
One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.
7 days ago
One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.
My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.
I feel the same way. Maybe it’s the ADHD, maybe I’m just dumb, but I cannot parse well the giant walls they tend to produce.
It’s melting my brain to read them all day. Our merge request descriptions are a mile long and so dense with jargon that it’s very difficult to figure out the important part of the changes.
They turned the english language into enterprise java and my train of thought is now a series of NullPointerExceptions
1 reply →
An LLM conversation is like handling clay. When I don't grok an answer I mold the LLM's approach to fit my level of mastery of the subject. It's one of the few interactions you can have in life where you can tell someone how to talk to you without considering how they feel about being ordered around.
That's interesting and actually the opposite of mine. I wonder if it's stack or methodology dependant? For reference I'm usually using cursor and opus4.6 and for a bigger piece of work:
- Start in ask mode - "I'm planning on doing X to achieve Y; are there any alternative approaches? What problems might I run into?"
- Chat for a bit and get the high level approach, switch to plan mode and ask for a nicely formatted plan
- What's kicked out is already in the rough shape of the discussion so far, so it's a case of following a nicely formatted doc through and highlighting sections of text and asking for clarification or changes
- Hitting "build" and then reviewing what's been done
For a new service I might spend an hour in ask/plan mode - but then it gets 95% of the build itself right first time.
Do you do the same with different results, or is there a different stack/methodology you go through?
I get a lot of this in design docs every time I give it a negative constraint:
[Suboptimal choice]
And here's why it's not suboptimal -- you said X sucks and notto do X, but this choice is not technically X, it's just really similar and shares that sucky property.
I've been in a lot of situations where I could step gpt5.x through a big refactor if I spoon feed it one type name at a time. If I let it try to do the whole thing at once it will refuse or get stuck in apply patch loops.
Planner / executor separation can make a huge difference in performance. LLMs are fantastic at coming up with a lot of elaborate narratives regarding what should be done. They are terrible about doing that prescribed work all at once. This impedance mismatch is best resolved with a simple role separation. Placing a shared collection of tasks between these roles is how you can decouple them. The executors need significantly more tokens than your planners to get the job done. It's probably in the range of 10-100x more for really complicated jobs with a lot of iterations through compiler feedback, sql provider errors, etc. This is why you can't do both things in the same context very well.
At that point I would rather just write the plan myself
Okay but that means you already know the plan since you are qualified to review it. So why not just tell it the plan yourself (0-shot) vrs having it guess and you review multiple times (n-shot). Wouldn't the former be more effective everytime?
[dead]