Comment by girvo
3 days ago
Being grafted onto the main model reduces layer duplication that you’d otherwise have: at least for Step and Qwen 3.6
3 days ago
Being grafted onto the main model reduces layer duplication that you’d otherwise have: at least for Step and Qwen 3.6
Step 2.7’s MTP seems broken (at least for ik_llama.cpp) where the draft model starts and ends in block 3 but ik_llama bails out looking for block 0 :(
Aw that’s a shame; I’m running the official llama.cpp on my Spark-alike, and it works great now. Proper triple head too which is what it is trained on, gets me up to 35-40tk/s decode