Comment by boredatoms

It also depends on the runtime, vllm is unbelievably slow at model loading compared to llama.cpp