Comment by hmokiguess
1 day ago
Maybe optimize the harness for less turns and less tokens to deliver real value as opposed to the turn taking token hungry so you can less load on your servers, oh, right, your entire bottom line is tokens.
1 day ago
Maybe optimize the harness for less turns and less tokens to deliver real value as opposed to the turn taking token hungry so you can less load on your servers, oh, right, your entire bottom line is tokens.
I don't understand this take. Ultimately, people pay for how much they're able to accomplish with the model, not raw token count. If Anthropic thought people could accomplish the same amount with fewer tokens, they'd adapt the harness to do that and then raise the cost of tokens to make more profit (or lose less).
I mean, it's a near daily experience for me that Claude tries to 'sudo pacman -S' some dependency multiple times before giving up and admitting that it's fundamentally impossible for it to run that command in the first place.
I could accomplish a lot more with those tokens if it just asked for the package to be installed.
I mean - the models are impressive but also they're not perfect by any stretch. The fact that they're improving means there's room to improve
Anthropic is not fully in control of the model.
Then why are fable form anthropic and the 5.6 sol line from openai so much more token efficient than other models? I really don't get this take at all - they're supply constrained right now, and there's literally no economic incentive to make each response take more tokens for the same output when we're in a market as intensely defined by induced demand/jevons paradox as this one. People are hitting their limits. If they make each turn take less tokens and each session take less turns, people will make more sessions.
I said optimize your harness, not your model. My comment was with regards to Claude Code. There are lots of ways to achieve the same task with less turns with different engineered harnesses around more efficient models.
Look at features they release, such as Dynamic Workflows, that spin up 100+ sub agents then reconcile the result.
Does that make sense now?
Yes? Dynamic workflows are incredibly powerful - especially for massive rote migrations. They're also particularly nice when you want to stretch an expensive model like fable and you can have it orchestrate a bunch of sonnet agents or whatever and get decent results without having every token burned be a fable token. For my usage style I find the best thing to be about dynamic workflows to be a sort of context management win. The orchestrator agent, if it was implementing everything, would end up deep into its context window where performance degrades steeply, and if you ran into issues early on the session you'd have to be concerned about whether whatever reasoning traces are in context are poisoning your later outputs, while if you're using subagents once you steer the model whatever is doing implementation is starting from fresh context with a new prompt so you aren't forced to kill and restart your session all the time. The way workflows are implemented - scripting with clearly defined stages also mean that the workflow will terminate eventually instead of running forever.
Now with that said, opensi's ultra mode is absolute trash - they should just have stolen Claude code's implementation - and there are clearly modes like max reasoning where they'll burn double the tokens to get another 10th of a percent performance to win benchmarks, which you should basically never be using. They're not making those versions at the expense of more efficient reasoning levels though and I don't mind that they exist (as long as they don't get made the default mode - openai - fix your ultra mode already).