Such a system is RAM bandwidth limited and not compute limited Switching to q4 from q8 would decrease the amount of data needing to be loaded by half. The token generation rate would nearly double. But generally if you can do q6 or q8 and you have enough RAM you really should. Even if it's slower.
any reason you went with q8 over q4? I'm wondering if q4 would run noticeably faster or not.
Such a system is RAM bandwidth limited and not compute limited Switching to q4 from q8 would decrease the amount of data needing to be loaded by half. The token generation rate would nearly double. But generally if you can do q6 or q8 and you have enough RAM you really should. Even if it's slower.