Comment by daft_pink
7 days ago
I feel the primary benefit of this Ollama Turbo is that you can quickly test and run different models in the cloud that you could run locally if you had the correct hardware.
This allows you to try out some open models and better assess if you could buy a dgx box or Mac Studio with a lot of unified memory and build out what you want to do locally without actually investing in very expensive hardware.
Certain applications require good privacy control and on-prem and local are something certain financial/medical/law developers want. This allows you to build something and test it on non-private data and then drop in real local hardware later in the process.
> quickly test and run different models in the cloud that you could run locally if you had the correct hardware.
I feel like they're competing against Hugging Face or even Colaboratory then if this is the case.
And for cases that require strict privacy control, I don't think I'd run it on emergent models or if I really have to, I would prefer doing so on an existing cloud setup already that has the necessary trust / compliance barriers addressed. (does Ollama Turbo even have their Trust center up?)
I can see its potential once it gets rolling, since there's a lot of ollama installations out there.
Me at home: $20/mo while I wait for a card that can run this or dgx box? Decisions, decisions.
Quickly test… the two models they support? This is just another subscription to quantized models.
it looks like the plan is to support way more models though. gotta start somewhere.