Comment by rafaelmn

You can run any open model on a shared API host via OpenRouter and pin to which host you want to go for the quant/privacy/etc. mix you care about. You can pay them directly if you don't want the OpenRouter overhead - but the convenience of switching, having one invoice, etc. is worth it IMO

It's not closed hosted models vs open local models, it's hosted open models vs local open models where the math doesn't work for local LLMs.

The only local inference use-case I can think of is porn generation (because most providers don't want to deal with it) and illegal shit like hacking to minimize the tracing.

And if you're super paranoid - but honestly giving sensitive info to LLMs in any scenario is a gamble.

If you game and can use your GPU I guess then it works as well but models that fit into a gaming GPU suck too much to bother IMO.