Comment by cobbzilla
5 days ago
I can understand (not necessarily defend) protectionism for products with nonzero manufacturing cost and some required physical size and such.
I am befuddled when the same policies are applied to products with marginally zero manufacturing cost, no real physical size, and such. Regulating ideas is hard folks.
The cost of those output tokens is not zero or marginal.
You could run the inference locally using the open weight models. Then you'd still get to use the models without sending money overseas
The first token on a new AI rig costs $X (full capex cost) then every token after that costs virtually zero. Over time the cost/token trends towards zero (modulo opex). That said, AI does have higher opex than general SaaS so it can’t get as close to zero.
But that’s kind of a different question, the running of some service. The “product”, the model, is a collection of files. The “manufacturing” required to add another customer is “send them the files” and has ~zero marginal cost.
There is a finite number of tokens that a rig will turn out over it's lifetime. Divide the cost of the rig plus electricity by that number and you have your cost per token. Yes providers can screw up on scheduling and end up paying more than they should, but that's not magic.
5 replies →
>I am befuddled when the same policies are applied to products with marginally zero manufacturing cost, no real physical size, and such.
Because it's not about perfect obstructionism, it's about adding red tape to businesses that end up relenting and going to a frontier model. If this gets these frontier models an extra few million dollars in exchange for greasing some palms, that's a business success.