Comment by zazibar
2 days ago
Good news, MTP support is already included in this release. Not sure why they haven't made this clearer.
2 days ago
Good news, MTP support is already included in this release. Not sure why they haven't made this clearer.
I don't see an MTP entry on Unsloth though. Maybe it's not available in a lower quant I need for my poor GPU.
It's included in the model gguf itself.
Maybe I'm not seeing the speed improvements because it's a dense model, and I'm comparing it to the 3.6 MoE with MTP. Likely, I'm attributing the speed I see to something else.
Honestly, if MTP is available, I don't know why it's so much slower here.