Comment by giancarlostoro
9 hours ago
On my 24GB RAM M4 Pro MBP some models run very quickly through LM Studio to Zed, I was able to ask it to write some code. Course my fan starts spinning off like the worlds ending, but its still impressive what I can do 100% locally. I can't imagine on a more serious setup like the Mac Studio.
Your limitation after prefill is memory bandwidth. A maxed out Studio has less than a single 3090 (really).
How is the output quality of the smaller models?
not good enough for coding anything more than simple scripts.
generally, the less parameters, the less knowledge they have.
what model were you using?