Comment by mickeyp
5 days ago
Shameful plug: my upcoming app perhaps?
Single-file download, fuss-free and install-less that runs on mac, windows and linux (+ docker of course.) It can run any model that talks to openai (which is nearly all of them), so it'll work with the big guys' models and of course other ones like ones you run privately or on localhost.
Unlike Claude Code, which is very good, this one runs in your browser with a local app server to do the heavy lifting. A console app could be written to use this self-same server, too, of course (but that's not priority #1) but you do get a lot of nice benefits that you get for free from a browser.
One other advantage, vis-a-vis Armin's blog post, is that this one can "peek" into terminals that you _explicitly_ start through the service.
It's presently in closed alpha, but I want to open it up to more people to use. If you're interested, you and anyone else who is interested can ping me by email -- see my profile.
>run any model that talks to openai (which is nearly all of them)
What does that mean? I've never seen any locally run model talk to OpenAI, how and why would they? Do you mean running an inference server that provides an OpenAI-compatible API?
Sorry, to clarify: OpenAI has an specification for their API endpoints that most vendors are compatible with or have adopted wholesale.
So, if your model inference server understands the REST API spec that OpenAI created way back, you can use a huge range of libraries that in theory only "work" with OpenAI.
> OpenAI has an specification for their API endpoints that most vendors are compatible with or have adopted wholesale
Worth clarifying that what the ecosystem/vendors have adopted is the "ChatCompletion" endpoint, which most models are under. But newer models (like codex) are only available under the Responses API, which the ecosystem/vendors haven't adopted as widely, AFAIK.