Comment by chamomeal

7 hours ago

If I could get the equivalent of GPT-4 running locally, that would cover like 95% of what I need an LLM for. Tweak this dockerfile, gimme a bash script. I guess the context probably isn’t sufficient for the agent stuff, but I’m sure more context-efficient harnesses will be coming down the line

2 comments

chamomeal

Reply
mark_l_watson  5 hours ago

I have an old Mac Mini with 32G of integrated RAM, and the following works for me for small local code changes:

ollama launch claude --model qwen3.6:35b-a3b-nvfp4

In addition to not having an integrated web search tool, one drawback is that it runs more slowly than using cloud servers. I find myself asking for a code or documentation change, and then spending two minutes on my deck getting fresh air waiting for a slower response. When using a fast cloud service I can be a coding slave, glued to my computer. Still, I like running local when I can!