Comment by bissellator
2 days ago
running locally is great, but I would wonder about the system requirements (I love my mac air but ollama on 8GB with an Intel i5 probably would be a little too much for the little thing). But having a toggle option would be great
Right? I feel like local model support is something that has strong ideological appeal and might influence someones feeling towards the app, but when it comes down to it most people will probably just use a cloud model for larger tasks unless they have beastly hardware. It's like how I have an Android in part because I might one day flash the ROM.
Ollama just exposes models via an OpenAI compatible endpoint though (I'm pretty sure), so adding that standard is probably a good idea. The prompts are a bit tuned for Gemini. I'd have to test how much that matters.