Comment by adamtaylor_13
5 hours ago
Ed is an interesting character. His financial analysis of the AI industry makes logical sense to me (though I am not knowledgeable enough to actually know if it is correct.) However, he seems to be so angry at AI in general, that he misses the obvious areas where LLMs are actually changing the State of the Art.
Coding seems to be one of the core use-cases for LLMs (as Simon Willison pointed out recently) and even if that's the only real use-case for LLMs, they're wildly useful. I do understand that useful != profitable and that's where I think Ed has a real point: until inference becomes much cheaper these companies cannot be profitable. Some mega-players will pay the API token price, but most will not.
I don't think whether "LLMs are actually changing the State of the Art" or not matters for anything he wrote.
If the AI companies need $X billion in revenue to stay afloat, it doesn't matter if 0.5% or 5% or 50% of that revenue is from transforming the State of the Art. It's 100% irrelevant: what matters is that, transformation or no, these companies won't have the income to pay their bills. And if they can't pay their bills, a whole lot of other companies can't either.
So again, transformation or no, it's still a house of cards waiting to collapse. The only thing that would change that is not more "transformation" ... it's a feature set that lets them multiply their current user base (or multiply how much they charge them) several times over.
He's got subscribers. Maybe the attitude is one he's found plays well with them.
I find it quite refreshing in some ways. Lots of people, when they start complaining about this or that aspect of this AI stuff, are wont to add in a little disclaimer that, despite all of the above, they actually really like AI and use it all the time. I assume this is to avoid the scenario of a bunch of pragmatic builders turning up and calmly shipping nuance in the comments (or whatever you call it these days when you get brigaded by a pile of angry keyboard warriors with chips on their shoulder) - and it sure is tiring having to wade through the equivocation.
That's a criticism that'd be hard to level at Zitron! Say what you like about the man, but he's unafraid to appear to take a side.
It's pretty likely that inference will get substantially cheaper. His argument is that for these companies to be profitable some very major and (pre 2022) unprecedented things have to happen. Which I tend to agree with, except I think they will happen, seeing as how they've been happening for a few years.
> until inference becomes much cheaper these companies cannot be profitable. Some mega-players will pay the API token price, but most will not.
This is often repeated but comes from ignorance mostly. You have * zero * reason to believe inference is costly other than just vibes. If you go by data and intuitions - the margins are high.
This kind of thinking really reinforces my belief that people have no idea and are using this whole [AI is not profitable and too costly] thing as a cathartic way to deal with immense progress.
We know that inference cost is very significant, as he shows for example in this piece.
https://www.wheresyoured.at/oai_docs/
However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.
Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.
I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.
No reason to doubt his journalistic integrity? He's not a journalist for starters. He's a PR flack who does PR for AI startups on the side while blogging on substack. There is every reason to doubt his journalistic integrity.
3 replies →