Comment by Tuna-Fish
1 day ago
The key AI labs are not public companies, they are at liberty to brag about their margins to potential investors in private.
1 day ago
The key AI labs are not public companies, they are at liberty to brag about their margins to potential investors in private.
And investors will leak such claims quickly enough that this reasoning cannot plausibly hide big secrets.
It's not a big secret. If you just do the math yourself, it's easy to compute that inference doesn't cost all that much. People just see all the capital investment going around and all the new data centers being built, see that it's spent on "AI", put two and two together and get a three, or "clearly serving AI requests costs an arm and a leg".
The 1 they were missing is that AI requires both training and inference, and training is by far the expensive part. And that in principle you can stop training at any point and keep using the models as they are. (But that means that if other companies keep improving their models, you'll be left behind...)
In contrast, inference is fairly cheap and all the providers have great margins on it. Eventually either investment in training stops having commensurate impact on model quality, and people stop doing that and instead concentrate on making inference faster and even more efficient. Or if that doesn't happen, things will get very weird very quickly.
The market already shows where it will go.
If you want frontier model you will pay more for inference to essentially fund the expensive training.
If you don’t need frontier model you will get dirt cheap inference, which eventually will approach the cost of electricity spent per token.
This is technically correct, but practically false.
They can't stop training as then the AI's knowledge will become out-of-date very quickly. Their knowledge stops the day you stop training.
1 reply →
> If you just do the math yourself, it's easy to compute that inference doesn't cost all that much.
Show us your work, then. If it's so easy to do, this should be a trivial request to accommodate, no?
4 replies →
I’m wary of “has not been leaked in a way that was picked up in public news” as proof or disproof of anything.
this is changing soon
Not really, how much of a public company are you when 5% of your capital is public ?
That doesn't matter for the legal requirements.
The short and only kind of wrong version is:
In the US, companies are not allowed to unfairly privilege some investors over others by giving them access to secret information that would let them judge the future prospects of the company. (Except in all the ways they can, but these usually involve some kinds of insider trading rules.) Private companies can handle giving out secrets to investors by literally writing and memo and mailing it to all their investors, if they want to give out some secrets to one of them.
Public companies cannot do that, even if they knew who all their investors were, but must instead consider every member of the public a potential investor, even if they don't already own the stock. Because of this, when public companies want to reveal material information about their future prospects, they must reveal it to everyone.
The percentage is irrelevant for this discussion. As soon as you’re public, you need to report detailed financial numbers.
10 replies →
Isn't there a limit on the public markets where if a company has less than a certain percentage of its ownership traded publicly then it is no longer a public company and therefore de-listed?
I remember hearing about a guy trying to squeeze out short sellers of his own company but ended up effectively taking his company private because he bought out like 95% of all the shares.
I wonder how that aligns to these small releases of stock for the public.