Comment by Zagitta

It's also useful for prefill in inference, VLLM sees up to 30% boost depending on the exact kernels and prefill size