Comment by deskamess

19 hours ago

Did DeepSeek come up with MTP? It was listed prominently in their recent paper as being carried forward from the previous release.

2 comments

deskamess

Reply
logickkk1  16 hours ago

i think this is mixing two separate ideas. MTP is the training-side piece. speculative decoding is the inference trick. DeepSeek V3 used MTP as an auxiliary loss. the 2022 Google paper is speculative decoding. now Google is combining them. https://arxiv.org/abs/2404.19737

  • deskamess  16 hours ago

    Oh... so MTP is not speculative decoding? The (T)oken (P)rediction made me think it was on the inference side. I shall read the paper.

    Edit: Ok, I understand now. You are saying that MTP has two aspects. 1) The training (for the mini-models to generate tokens), and 2) The actual speculative decoding implementation on the inference side (which uses those trained mini-models).