Comment by albinstman
17 hours ago
I’ll look into adding harnesses. The benchmark sources usually don’t contain harness so the problem is limited data. What models would you like to see?
17 hours ago
I’ll look into adding harnesses. The benchmark sources usually don’t contain harness so the problem is limited data. What models would you like to see?
For example you might add Strata & Qwen 3.8 Flash Next with some quantizations. Nowadays it is a go to selection and very sought after MoE model.
New model/hardware benchmarks is now live
Qwen 3.8 flash next is already in the works and should be live soon.
He's exagerating, I hope. The only bottleneck a harness attachs is pre-context fill and whatever tooling is installed. A proper harness reduces these to negligable startup costs.
Unless of course he's driving some weird lambo that dumps 200k of context into their chats. Who knows, I feel like we're on a yellow submarine.
Exagerating? I setup llm endpoints on corporate level in a company on H200 gpus, that serves hundreds of software engineers, I have observed differences, even on different versions of nvidia drivers. I shared my day to day experience. It is more and more important if you have very limited VRAM like 24gb on a 4090 RTX desktop or RTX 5090 Laptop gpu. I set up and use llms on my personal devices also, when I can't use cloud models on classified company materials. Here is an example problem what I daily report and deal with.
https://github.com/ggml-org/llama.cpp/issues/27330
I can believe different drivers make a difference, but that wasn't the point under debate
1 reply →