Comment by jedbrooke

nope just my normal m2 mac mini. I bought it just as a normal computer to do my taxes and whatever, so it’s mind blowing that I can run this kind of AI workload on it. Well, “run” might be generous, it gets like 3-5tok/s

I’m working on a setup that’s more geared towards running tasks overnight so the slow tok/s doesn’t matter as much