Even at just the electricity cost openrouter will be both
1) Roughly break-even to a little bit cheaper per token cost
2) Much, much, faster
So the cost of the mac barely even matters, it's just an extra cost beyond.
Sure, data center providers can pay lower rates.
The point of this article is that LLMs at home really don't make a ton of sense, unless you are willing to pay through the nose for privacy. There is absolutely no cost saving to be had.
If you're looking at your own datacenter as a larger corporate client, that could change.
There are also some providers that will contractually keep your data private, like AWS Bedrock or parts of Google/Azure (I don't know their stack names).
AWS even has AWS Secret Region and AWS Top Secret Region if you want to use LLMs on classified data.
You have to value privacy at a roughly absurd level to not want to use LLMs run efficiently at scale by someone else. For the home user, just the extra efficiency produced by batching requests from a large number of users in a datacenter in a real win.
Some of these companies are even selling tokens below cost to get marketshare. If someone will sell you a service for a dollar bill or three quarters, why wouldn't you take the three quarters?
It can get quite complex, for me the feeling of "I built this", and then watch as it makes stuff from the power generated on my solar panels is also worth a lot to me. The sheer coolness of it.
Not always. The calculations take its useful life expectancy as an input. If they estimate it correctly you have highly likelihood of it breaking, burning out or being woefully out of date by the end. At the 10 year window you are looking at losing support for security updates.
So if you are lucky you might end up with something that still runs but most folks won't find it particularly useful
Real estate is generally a "good" investment as it's considered a relatively safe way to get significant leverage. 5x leverage in the case of a 20% deposit, or even up to 20x leverage with countries that allow for 5% deposits (New Zealand).
In addition, the interest payments almost always end up being near the rent the owner would have paid, so mortgage payments are higher, but that increase is generally (and quickly becomes) principal while being able to counteract inflation of rent.
Articles like that still miss a bit of the nuance. Imagine having your house paid for, and you grow old and you have no rent to pay. Yes, you could have invested but likely you would have spent some of that money on something else, or your investments might have not worked out so well, or any other reason. Human reasons, to be specific. Owning property is like a lock.
It is very close to universal truth, aside some small areas with very warped market.
Even if you move out after 5 years, you still own the place and can rent it out and then it pays for itself, to skip the cost of selling it back to market
It never fails, there's always someone who trots this thing out. We had bought our house, and then had to move and decided to rent. I was APPALLED that they wanted me to fill out an APPLICATION form, where they would decide my worth, and let me know if we would be allowed to live there. When buying a house, my cash was as good as anyone elses'. And then the management company would come inside my house to inspect that I wasn't running a meth lab or something. Thankfully that only lasted two years. I will never rent again. Majority owner-occupied neighborhoods have different characteristics as well.
Except one day the hype will catch up to reality that was always true, people will realize their $20,000 Mac is has less utility as a "way to learn AI" than some kids 3090 fortnite machine, and it'll be back to below MSRP.
Even at just the electricity cost openrouter will be both
1) Roughly break-even to a little bit cheaper per token cost 2) Much, much, faster
So the cost of the mac barely even matters, it's just an extra cost beyond.
Sure, data center providers can pay lower rates.
The point of this article is that LLMs at home really don't make a ton of sense, unless you are willing to pay through the nose for privacy. There is absolutely no cost saving to be had.
If you're looking at your own datacenter as a larger corporate client, that could change.
There are also some providers that will contractually keep your data private, like AWS Bedrock or parts of Google/Azure (I don't know their stack names).
AWS even has AWS Secret Region and AWS Top Secret Region if you want to use LLMs on classified data.
You have to value privacy at a roughly absurd level to not want to use LLMs run efficiently at scale by someone else. For the home user, just the extra efficiency produced by batching requests from a large number of users in a datacenter in a real win.
Some of these companies are even selling tokens below cost to get marketshare. If someone will sell you a service for a dollar bill or three quarters, why wouldn't you take the three quarters?
> If someone will sell you a service for a dollar bill or three quarters, why wouldn't you take the three quarters?
Because one day they'll send you an email informing you the new rate is $1.50, and if you missed the email, that's not their problem.
Plus your privacy.
Minus opportunity cost and depreciation
It can get quite complex, for me the feeling of "I built this", and then watch as it makes stuff from the power generated on my solar panels is also worth a lot to me. The sheer coolness of it.
Not always. The calculations take its useful life expectancy as an input. If they estimate it correctly you have highly likelihood of it breaking, burning out or being woefully out of date by the end. At the 10 year window you are looking at losing support for security updates.
So if you are lucky you might end up with something that still runs but most folks won't find it particularly useful
Yea this; it’s the same reason why mortgaging is cheaper than renting
This is far from a universal truth: https://www.nytimes.com/interactive/2024/upshot/buy-rent-cal...
Real estate is only a clearly good investment if you ignore opportunity cost.
Real estate is generally a "good" investment as it's considered a relatively safe way to get significant leverage. 5x leverage in the case of a 20% deposit, or even up to 20x leverage with countries that allow for 5% deposits (New Zealand).
In addition, the interest payments almost always end up being near the rent the owner would have paid, so mortgage payments are higher, but that increase is generally (and quickly becomes) principal while being able to counteract inflation of rent.
9 replies →
You also need to pay close attention to rent vs purchase ratios. A lot of cities are cheap to rent but expensive to buy (eg beijing 10 years ago).
3 replies →
Articles like that still miss a bit of the nuance. Imagine having your house paid for, and you grow old and you have no rent to pay. Yes, you could have invested but likely you would have spent some of that money on something else, or your investments might have not worked out so well, or any other reason. Human reasons, to be specific. Owning property is like a lock.
6 replies →
It is very close to universal truth, aside some small areas with very warped market.
Even if you move out after 5 years, you still own the place and can rent it out and then it pays for itself, to skip the cost of selling it back to market
It never fails, there's always someone who trots this thing out. We had bought our house, and then had to move and decided to rent. I was APPALLED that they wanted me to fill out an APPLICATION form, where they would decide my worth, and let me know if we would be allowed to live there. When buying a house, my cash was as good as anyone elses'. And then the management company would come inside my house to inspect that I wasn't running a meth lab or something. Thankfully that only lasted two years. I will never rent again. Majority owner-occupied neighborhoods have different characteristics as well.
3 replies →
Except one day the hype will catch up to reality that was always true, people will realize their $20,000 Mac is has less utility as a "way to learn AI" than some kids 3090 fortnite machine, and it'll be back to below MSRP.
Where is a Mac above MSRP?