Comment by natrys
16 hours ago
I think we don't have a good draft model for better speculative decoding yet (e.g. DFlash 2). Once we do, it will be faster.
16 hours ago
I think we don't have a good draft model for better speculative decoding yet (e.g. DFlash 2). Once we do, it will be faster.
It very well could be faster, but right now it isn’t.