Comment by SwellJoe
19 hours ago
Even at 30 t/s, 3.8 thinks so long, even on medium, it still takes 3x or more longer than any cloud model, in my testing.
19 hours ago
Even at 30 t/s, 3.8 thinks so long, even on medium, it still takes 3x or more longer than any cloud model, in my testing.
I've got an M1 Max 64GB too. It's just not an LLM-class workstation. Give it a year and buy an M7 and you'll be laughing. Right now is a really bad time to invest in anything - using the cloud is the cheapest option, especially for open weight models.