Comment by Der_Einzige
2 days ago
Frontier model labs don't know anything about good sampling settings, and why should they? Their belief is that "sampling doesn't matter, we can simply scale our model and improve logprobs quality that way"
Ignore whatever nonsense qwen says about what settings to use. Turn off top-p and top-k. Turn on top-n-sigma and keep it at its defaults.
See repetitions that you don't like? Ignore the "repetition, frequency, presence" penalties and turn on DRY (don't repeat yourself).
Want more creativity? Temperature is okay but XTC (eXclude Top Choices) is better.
Llama.cpp quietly has had far advanced samplers compared to the shit stuff Qwen et al recommends for years.
No comments yet
Contribute on Hacker News ↗