Comment by turblety
3 months ago
Are you running the full 65GB model on a MacBook Pro? What tokens per second do you get? What specs? M5?
3 months ago
Are you running the full 65GB model on a MacBook Pro? What tokens per second do you get? What specs? M5?
If they're running 120B on a M5 (32GB max of memory today), I'd like to know how.
Probably an M4 which has up to 128GB currently
I am running the full model on an 128GB M3 Max.
On an m4 pro 128gb: 75 t/s.
Caveat: That's just for the first prompt.