Comment by jascha_eng
1 day ago
Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that.
I haven't really found a good way to solve this other than:
1. Produce an initial PR fulfilling all the requirements I knew at the start
2. Chat with the model about any weird snippets I notice and talk through alternatives
3. Simplify anything that I think is overengineered or plain unncessary
Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts.
In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?
For some frontier models like Fable 5 it doesn't matter, but for models less trained on long horizon tasks it very useful.
It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.
Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?
I feel like "do X untill tests are green" is sort of the prototypical /goal case.
Adding a new lint rule that a lot of code violates.
2 replies →