Maybe I'm not seeing the speed improvements because it's a dense model, and I'm comparing it to the 3.6 MoE with MTP. Likely, I'm attributing the speed I see to something else.
Honestly, if MTP is available, I don't know why it's so much slower here.
Maybe I'm not seeing the speed improvements because it's a dense model, and I'm comparing it to the 3.6 MoE with MTP. Likely, I'm attributing the speed I see to something else.
Honestly, if MTP is available, I don't know why it's so much slower here.