Comment by a12k
1 year ago
Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route.
When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs, which imo increases the sketchiness.
It's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?
It severely over-permissions itself on my Mac.
Can you please elaborate? How are you running ollama? I just build it from source and have written a shell script to start/stop it. It runs under my local user account (I should probably have its own user) and is of course not exposed outside localhost.
I have never had it request any permissions
Just install it from Brew and run the service in a separate terminal tab.
Isn't there a clause in MIT that says you're required to give credit? Also, I didn't know a YC company which started it: https://www.ycombinator.com/companies/ollama.
The project existed in the open source and then subsequently the creators sought funding to work on it full time.
1 reply →
> But why run it in a VM?
Because you don't execute untrusted code in your machine without containerization/virtualization. Don't you?
The question was asking why it’s untrusted code, not why you run untrusted code in a VM.
There are a lot of open-source tools that we have to trust to get anything done on a daily basis.
Every single day. There's just too much good software out there, and life is too short to be so paranoid.
Care to elaborate what "sketchy" refers to here?
> but VMs are pretty easy so just went that route.
Don’t you need at least 2 GPUs in that case and put kernel level passthrough?
I don’t use GPU. Works fine, but the large Mixtral models are slow.
i pass through my dGPU to VM and use iGPU for desktop
ollama advertising llama.cpp features as their own is very dishonest in my opinion.
That’s the curse and blessing of open source I guess? I have billion dollar companies running my oss software without giving me anything - but do I gripe about it in public forums? Yea maybe sometimes but it never helps to improve the situation.
It's the curse of permissively licensed open source. Copyleft is not the answer to everything but against companies leeching and not giving back it is effective.
1 reply →
Are they a wrapper with a similar name? You, like I, do gripe in public forums.
2 replies →
Welcome to open source.