Comment by mandeepj

3 days ago

> I use local LLMs on my Mac Mini.

which ones do you use?

3 comments

mandeepj

Reply
LeBit  3 days ago

I use the following models:

  - Qwen3.5 9B
  - Qwen3.6 35B A3B
  - Qwen3.8 27B
  - Gemma 4 26B A4B
  - Gemma 31B
  - Muse Glimmer 30B

I have 48G.

The MacMini is used solely for inference. llama.cpp + llama-swap.

People are saying local models are crap and serves no purposes.

I use them to help me spell check, write emails, write text messages, write JIRA tickets, write PR comments, etc.

I also use them in coding agents to complete different tasks.

I find them quite useful!

  • mandeepj  2 days ago

    Thanks! I have two mac minis: one with 8 gb and the other with 24 gb (left with 17 gb useable ram); I can't run any open model on them. So, ordered a Mac 5 pro with 128 GB. Can't wait for its arrival.

    • LeBit  2 days ago

      128G of RAM? I envy you.

      You will be able to run some pretty impressive models locally. Good for you.