Is there a secure way to use GLM without spending $10K’s for local HW? I “only” have a 128GiB inference machine, and don’t really trust anthropic not to steal my IP over time.
I see no reason to trust Z.ai more than other vendors.
Kind of, you need at least 256 gb of vram and 24-40 gb of vram to run the 2bit quantization, because it's a moe you just need the expert to fit in vram to get significant improvement over a pure CPU setup. At 2bits though expect significant quality loss.
Is there a secure way to use GLM without spending $10K’s for local HW? I “only” have a 128GiB inference machine, and don’t really trust anthropic not to steal my IP over time.
I see no reason to trust Z.ai more than other vendors.
Kind of, you need at least 256 gb of vram and 24-40 gb of vram to run the 2bit quantization, because it's a moe you just need the expert to fit in vram to get significant improvement over a pure CPU setup. At 2bits though expect significant quality loss.
2bits is a joke for serious work. You'd be better with Qwen3.6 under 30G probably.
But there are EU only providers for GLM5.2. For example tensorx. Depending on your definition of "secure" it may be acceptable.
1 reply →
Ollama Cloud has a $20 a month subscription. They say they retain 0 information. And rather than token based billing, it's GPU time billing.
You'd need to multiply that $10k by 8 minimum.
Why? 4 x DGX sparks should be enough. That's way less than $80k.
4 replies →
Neither does chatgpt. And is the harness lock-in such massive problem that you would pay 20x more?