Comment by ssvegeta
1 day ago
I'm seeing this infinite loop behavior during reasoning as well, for the exact same quant from Unsloth. Set presence penalty to 1 and the issue seemly went away, but having a penalty that high worries me for coding tasks. Will try the bartowski one now.
No comments yet
Contribute on Hacker News ↗