Comment by sarjann
4 days ago
Why use electricity alone for open models? Surely you’d want to spread the cost of hardware over the period too?
4 days ago
Why use electricity alone for open models? Surely you’d want to spread the cost of hardware over the period too?
I think the key there is “hardware you already own.”
If you own a graphics card you bought for gaming or a laptop you bought for doing schoolwork there is $0 in cost of local AI tokens, because 100% of the cost was assigned to doing other things.
It's bad accounting to only look at marginal cost and ignore depreciating hardware asset costs.
God, I'd love to have more local VRAM but the card costs are out of control.
Based on these costs I'd almost expect the amount of gaming graphics memory to go down over the next few years putting more stress on running those local models.