Comment by MadameMinty

12 hours ago

s/understate/overstate/ I mean, I get up to 90 tok/s on consumer hardware (16 GB VRAM + 64 GB RAM) using IQ3_S, which so far is holding up quite well.

0 comments