Comment by nayroclade
12 hours ago
Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.
In single-turn use, yeah, but across dozens of turns there's probably value in optimizing the output.
Btw your point lands just as well without "Cute idea, but" https://odap.knrdd.com/patterns/condescending-reveal
I didn't mean it as condescending. I meant it literally is cute: A neat idea that is quite cool in its execution.
Good point and it's actually worse than that : the thinking tokens aren't affected by this at all (the model still reasons normally internally). Only the visible output that gets compressed into caveman... and maybe the model actually need more thinking tokens to figure out how to rephrase its answer into caveman style
Grug says you can tune how much each model thinks. Is not caveman but similar. also thinking is trained with RL so tends to be efficient, less fluffy. Also model (as seen locally) always drafts answer inside thinking then output repeats, change to caveman is not really extra effort.