Comment by swozey

Anyone have a good guide on properly setting up/optimizing local llm stuff like this?

I've been using omlx and qwen for almost a year but have bounced around clients a bunch, and it seemed like everything was specific to claudes style of config layout, so I've been putting all of my skills/agents/md files in my ~/.claude as a catchall for bouncing between pi.dev/claude/vscode/etc. and just seeing what happens. I really haven't used claude itself much so reasonable but I didn't know it didn't look at AGENTS.md, for instance. Also using some memory/kb system that puts $myKB.md in directories to pre-fill context by project/workspace.

I'm currently using oh-my-pi but in the quest for optimization and token trying to get better than 30t/s on my m1 max 64gb (qwen3.6-a35b) I probably need to spend some time just making pi base into what I need and not the opinionated omp setup I have that probably makes the initial context larger than it should be.

I'm between work and can't afford the $100+ frontiers but it does get really frustrating spending hours/days tweaking this stuff to almost no benefit sometimes. When I do get to use a frontier it's such a nice break from fixing things. The local llm stuff can definitely be a bit frustrating right now and zap the energy I have for work out of me when it goes awry.