Comment by kolinko

60tok/s prefill or tok gen? If you use gpu and keep it in vram then I you most likely mean token generation, nit prefill.