← Back to context

Comment by a11r

7 hours ago

A negative datapoint - We have been evaluated prior Deepseek Flash models (both the original 4.1 flash and the smaller 4.0 flash) and they have been overthinkers. Our workload is well defined but moderately difficult coding tasks and among open weight models GLM5.3 flash performs best with Qwen3.8 flash next a close second. DeepSeek 4.1 flash fails at lower thinking levels and spends a huge amount of tokens (an order of magnitude more) at the highest thinking levels where it is able to successfully complete the tasks. I will evaluate this newest release since they specifically claim "the model reduces task execution time and output length", but keeping my hopes low.

v4 was a worse overthinker. 4.1 mostly fixed that. It still prefers to plan everything before making a single file change.

If I'm impatient I switch to Luna for a single turn just to get it started and let DS continue.