Comment by layla5alive
7 hours ago
This tool has very tiny quants available for each model, and defaults to the tiny quants, so I'm not so sure you... tried it?
7 hours ago
This tool has very tiny quants available for each model, and defaults to the tiny quants, so I'm not so sure you... tried it?
Ya but the library matters as well and what will get 20tps on one might get 30-40tps on another, especially with finicky cards like Intel arc.