Comment by TheDong
8 hours ago
You can buy a roughly $40k gpu (the h100) which will cost $100/mo in electricity on top of that to get about 30-80% the performance of OpenAI or Anthropic frontier models, depending what you're doing.
Over 5 years, that works out to ~$45k vs ~$10k, and during that duration, it's possible better open models will come available making the GPU better, but it's far more likely that the VC-fueled companies advance quicker (since that's been the trend so far).
In other words, the local economics do not work out well at a personal scale at all unless you're _really_ maxing out the GPU at close to 50% literally 24/7, and you're okay accepting worse results.
As long as proprietary models advance as quickly as they are, I think it makes no sense to try and run em locally. You could buy an H100, and suddenly a new model that's too large to run on it could be the state of the art, and suddenly the resale value plummets and it's useless compared to using this new model via APIs or via buying a new $90k GPU with twice the memory or whatever.
This feels like it should be state infrastructure, the way roads, railroads and the postal system are.
This feels like a market which hasn't settled into long-term profitability and is being subsidized by investors.
And who is doing the research on this, training the models, and building new frontier models in your version of the world?
Note that the (edit: US) postal system is a for-profit system.
Given the trends of the capitalist US government, which constantly cedes more and more power to the private sector, especially google and apple, I assume we'll end up with a state-run model infrastructure as soon as we replace the government with Google, at which point Gemini simply becomes state infrastructure.
> Note that the (edit: US) postal system is a for-profit system.
That's not correct. If USPS makes more revenue than their expenses for a year, they can't pay it out as profits to anyone.
It's true that USPS is intended to be self-funded, covering it's costs through postage and services sold, and not tax revunue. That doesn't mean there's profit anywhere.
> Note that the (edit: US) postal system is a for-profit system.
Pricing in the US postal system is not based on maximizing profit. Ths US postal system is not a for-profit system, at all. It is a delivery system (more or less) that happened to start turning a profit (2006) until PAEA. After that, the next time it made a profit was 2025.
The USPS is self funding, not for-profit. The difference is both significant and consequential.
> Note that the postal system is a for-profit system.
That depends on the country in question :-)