Comment by LogicFailsMe

2 days ago

No barrier to entry whatsoever? Backprop on the speculative decoding weights during inference to improve their accuracy on a per application basis?

Cool hack though, kudos. Wonder if they can make Groq or Cerebras do the same thing?