Comment by techjamie
1 day ago
In my experience, Kimi just tends to think a lot, with the main thing that takes up a lot of space is it constantly second-guessing itself. I've watched it do paragraph after paragraph of "Wait, actually..." while it stumbled and used a ton of tokens on one small detail of what it was asked to do. Though I also gave GLM 5.2 a task to port some JS code to Python to test it, and in my experience it doesn't second guess as bad as Kimi does, but it really did there. It kept doing web searches and second guessing tons of tiny little things, using $0.25 of API spend in total to port about ~50 lines of JavaScript. It did produce an error the first run, but on second run it gave me a program that ran.
I gave Claude Code/Fable the same task and it took significantly less time, but also stumbled on the same error as GLM. I didn't have it fix it though. I was mostly interested in timing differences.
I do like open models where I can, but I'm really hoping they get trained to second guess less. Or maybe I just need to prompt them differently. I'm not sure.
Interesting. Could you not tell it to not second-guess itself (on minor issues)?
Fighting against the weights is often a lost cause. For any model.