Comment by cpburns2009
1 day ago
Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.
1 day ago
Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.
This is the default value!
Llama.cpp defaults to 1.0 (disabled) and so does vLLM. It looks like only ollama defaults to 1.1.