I suspect some of the issue id that some harnesses are over-optimized for particular models and their preferences (tool calling, instructions to soften their deficiencies etc).
Pi is much more minimalist - probably a fairer point of comparison.
A different suspicion of mine is that some people over-specialize in a given model - or maybe become lazy with their prompts or suffer from skill issues.
Fwiw - I generally maintain a specs/ folder as I code.
I never use “plan” mode - I just tell the LLM to make no code changes, but discuss design with me.
At some point I am happy (I typically ask it to summarize and write the actual spec), I review; correct misunderstandings, ask for follow-up questions, we incorporate the additional details into the spec and move on.
I often have TODO’s/tasks in those specs too and I regularly update progress on them. It also happens that I ask the LLM to review my code (actual) against the spec and search for differences- we then resolve them. Sometimes by modifying the code; sometimes by modifying the spec.
For starters, I write an overview spec - nail down the big concepts and architectural choices at a high level.
Moderately complicated facets of the application get their own spec - we write these as and when it gets relevant.
I think it helps the model a lot because I can refer to specs I feel relevant in drafting new specs or when solving tasks. And LLMs are generally better at proactively consulting these specs when getting an overview of the application and its design ahead of implementation.
Course not :)
I use pi (pi.dev).
I suspect some of the issue id that some harnesses are over-optimized for particular models and their preferences (tool calling, instructions to soften their deficiencies etc).
Pi is much more minimalist - probably a fairer point of comparison.
A different suspicion of mine is that some people over-specialize in a given model - or maybe become lazy with their prompts or suffer from skill issues.
Fwiw - I generally maintain a specs/ folder as I code.
I never use “plan” mode - I just tell the LLM to make no code changes, but discuss design with me.
At some point I am happy (I typically ask it to summarize and write the actual spec), I review; correct misunderstandings, ask for follow-up questions, we incorporate the additional details into the spec and move on.
I often have TODO’s/tasks in those specs too and I regularly update progress on them. It also happens that I ask the LLM to review my code (actual) against the spec and search for differences- we then resolve them. Sometimes by modifying the code; sometimes by modifying the spec.
For starters, I write an overview spec - nail down the big concepts and architectural choices at a high level. Moderately complicated facets of the application get their own spec - we write these as and when it gets relevant.
I think it helps the model a lot because I can refer to specs I feel relevant in drafting new specs or when solving tasks. And LLMs are generally better at proactively consulting these specs when getting an overview of the application and its design ahead of implementation.