Comment by sieve

I do not (generally) trust benchmarks. I only trust what a model does with MY code.

Forget DS. I asked MiMo 2.6 yesterday to explain ML/LLMs to me succinctly and the pointed it at Karpathy's micrograd code. It produced a C implementation called `xor_mlp`, a tiny model that learnt how `xor` worked. I then asked it to produce a model that can play tictactoe without losing (mostly). It did. It supervised the training process and produced a compiled version with multiple switches. The pi-dev session is still running, so here are actual stats

↑45k ↓35k R1.0M CH99.4% $0.019 4.2%/1.0M (auto) - (opencode-go) mimo-v2.6-flash • high

And here is Luna on the same workflow (I had to poke and prod a bit to get what I wanted):

↑141 ↓34k R1.0M W43k CH95.3% $0.072 4.2%/1.1M (auto) (opencode-go) gpt-5.6-luna • high

I expect similar results from DS41F/MS13. Closer to MiMo costs than Luna.

So the "significantly cheaper" thing may not really hold, more so when Luna has to actually read my codebase to do the stuff that I want rather than rely on world knowledge. The 8-10x cache read cost differential itself will kill the token budget.