Comment by westpfelia

I recommend trying pi.dev as your agent harness for local models. In my experience it has been the sweet spot of functionality (which you can and should extend with plugins) vs performance (OpenCode just swamps local models on my hardware).

you can try ollama, omlx or llama.cpp for instance to download a model and get an inference server running locally. They expose „open ai compatible“ endpoints, so you can configure almost any harness to use them.