Comment by ilaksh
9 hours ago
Have you thoroughly tested a slightly simpler workflow with one less stage but a frontier model doing the initial planning? Luna is amazing but it's a deliberate trade off designed for speed and cost savings. You can also give them a rule about speaking clearly.
The idea behind using Luna was cost-optimization.
I wrote a Pi extension called "model-escalation" which lets the model request an automatic switch to a smarter model for the next turn if it thinks the problem it is struggling with is too complex or if the user explicitly asks it to do so.
With this extension in hand, I could split the work between Luna and Sol, resulting in a ~95%-5% split. I don't have to switch to Sol when review time comes, Luna does it automatically. Sol tells its remarks, then Luna gets the control back and continues the back&forth (if there are still complaints).