Comment by simianwords
4 hours ago
> until inference becomes much cheaper these companies cannot be profitable. Some mega-players will pay the API token price, but most will not.
This is often repeated but comes from ignorance mostly. You have * zero * reason to believe inference is costly other than just vibes. If you go by data and intuitions - the margins are high.
This kind of thinking really reinforces my belief that people have no idea and are using this whole [AI is not profitable and too costly] thing as a cathartic way to deal with immense progress.
We know that inference cost is very significant, as he shows for example in this piece.
https://www.wheresyoured.at/oai_docs/
However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.
Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.
I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.
No reason to doubt his journalistic integrity? He's not a journalist for starters. He's a PR flack who does PR for AI startups on the side while blogging on substack. There is every reason to doubt his journalistic integrity.
The PR-thing was always openly communicated by him and is not some secret or gotcha. It's essentially "fleecing the boosters", which I fully approve of and do similarly myself.
I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.
It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.
So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.
If he is right, the numbers will show it.
2 replies →