Comment by lompad

3 hours ago

We know that inference cost is very significant, as he shows for example in this piece.

https://www.wheresyoured.at/oai_docs/

However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.

Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.

I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.

4 comments

lompad

Reply
remich  3 hours ago

No reason to doubt his journalistic integrity? He's not a journalist for starters. He's a PR flack who does PR for AI startups on the side while blogging on substack. There is every reason to doubt his journalistic integrity.

  • lompad  2 hours ago

    The PR-thing was always openly communicated by him and is not some secret or gotcha. It's essentially "fleecing the boosters", which I fully approve of and do similarly myself.

    I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.

    It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.

    So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.

    If he is right, the numbers will show it.