Comment by Der_Einzige
9 hours ago
Still way behind on LLM sampler support compared to llama-cpp. Where's support for top-n-sigma? for DRY? for XTC? C'mon guys!
9 hours ago
Still way behind on LLM sampler support compared to llama-cpp. Where's support for top-n-sigma? for DRY? for XTC? C'mon guys!
Feel free to submit a pull request
I did and no one did review despite me begging on the vllm slack!
Yes, I think this project is a good example of how complex domains aren't simply solvable just by pointing ever more intelligence at the system, even when the code is all open source and the issues are all well defined. I say this as a contributor!