Comment by NitpickLawyer
8 hours ago
> 2 Mac Studios at 21tokens/s or 4 Macs at 30tokens/s
Keep in mind that most people posting speed benchmarks try them with basically 0 context. Those speeds will not hold at 32/64/128k context length.
8 hours ago
> 2 Mac Studios at 21tokens/s or 4 Macs at 30tokens/s
Keep in mind that most people posting speed benchmarks try them with basically 0 context. Those speeds will not hold at 32/64/128k context length.
No comments yet
Contribute on Hacker News ↗