Comment by busfahrer
1 day ago
Great, thanks! :-) and to mirror another poster: what kind of prompt parsing (prefill) speed do you get for that model? Also how is the speed for the 27B model?
1 day ago
Great, thanks! :-) and to mirror another poster: what kind of prompt parsing (prefill) speed do you get for that model? Also how is the speed for the 27B model?
35B: 1300-1800 t/s on both Q4 and Q6.
27B: give me 20 minutes
Thank you, good sir!