Comment by minimaxir
7 hours ago
For the problems I work on with GPT 5.6 Sol and the checks and balances I have in place, I estimate:
- 80% of prompts get everything correct and are confirmed correct with manual validation
- 19% of prompts make a minor mistake based on an ambiguity of the original prompt (user error not LLM error), but then reliably fixed in a followup prompt
- 1% of prompts causes more problems than it solves and is more pragmatic to just revert
For 99% good output, there isn't much of a dopamine rush when there is good output. The dopamine rushes are for the <1% odds.
From the other replies on this post, I suspect no one believes me, but I am offering these numbers in good faith.
I think those of us who are using AI consistently believe you and understand. I'd say roughly the same thing about Claude in terms of numbers.
I think many people who don't believe you just haven't built-up the kind of prompt history & MCP / CLI tooling etc that lets you get to the point where things work at that level of accuracy.
Hope it helps to know that at least some of us here understand and are seeing the same thing. And if it's anything like my experience with Fable, "always be more ambitious". The capabilities of the models are often limited only by what you're brave enough to ask for. I keep finding I'm not ambitious enough.