Comment by billylo
5 days ago
Windows and macOS does come with a small model for generating text completion. You can write a wrapper for your own TUI to access them platform agnostically.
For consistent LLM behaviour, you can use ollama api with your model of choice to generate. https://docs.ollama.com/api/generate
Chrome has a built-in Gemini Nano too. But there isn't an official way to use it outside chrome yet.
Do you know what it’s called, at least on Windows? I’m struggling to find API docs.
When I asked AI it said no such inbuilt model exists (possibly a knowledge date cutoff issue.)
https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica
Yes. I am not aware of a model shipping with Windows nor announced plans to do so. Microsoft’s been focused on cloud based LLM services.
This thread is full of hallucinations ;)
These are the on-device model APIs for apple: https://developer.apple.com/documentation/foundationmodels
Windows doesn't?
Is there a Linux-y standard brewing?
Each distro is doing their own thing. If you are targeting Linux mainly, I would suggest to code it on top of ollama or LiteLLM