Comment by grommz
4 hours ago
Imagine you're a mid sized company and you can host this model locally. Suddenly there are zero reasons to pay a single red cent to the bloodsucking American AI cartel.
4 hours ago
Imagine you're a mid sized company and you can host this model locally. Suddenly there are zero reasons to pay a single red cent to the bloodsucking American AI cartel.
Can you host the model for a lower cost per token than you'd pay Anthropic or OpenAI for a similar level of intelligence? I doubt you're beating their efficiencies of scale.
I dont have estimates on the cost of running models, but I think openai and anthropic are running on subsidized prices. At actual prices it might be worth it in the future.
how is this idea still so persistent? The fact people are able to run open models with about the same performance at 1/10th the cost should make it glaringly obvious that Anthropic has massive inference margins at api pricing.
1 reply →
No, and the reason is simple: Usage is bursty and if you don't maximize usage of the hardware you're going to lose on price.
Ok you can host this model once. What if I want a dozen subagents? Ok you can host it 12 times at once. What if we go a whole week only using max 4 at a time? Etc etc. The limits imposed by self-hosting might be bearable for a variety of reasons, but it's going to be more expensive and less convenient/useful.
Whether it is "open" or not seems to be in question. While it was initially called an "open" model, it seems that "open" mentions have been scrubbed from website.
hardware, electricity cost and other extra time consuming deployment, are they joke to you? ROI needs to positive otherwise open models have still BIG COST.