Comment by brandall10
7 months ago
We’ve seen $T+ scale impacts from AI over the past few years.
You can argue the distribution is hard to pin down (hence my note on risk), but let’s not pretend there’s zero precedent.
If it turns out to be another winter at least it will have been a fucking blizzard.
The distribution is merely tricky to pin down when looking at overall AI spend, i.e. these "$T+ scale impacts."
But the distribution for individual researcher salaries really is pure guesswork. How does the datapoint of "Attention Is All You Need?" fit in to this distribution? The authors had very comfortable Google salaries but certainly not 9-figure contracts. And OpenAI and Anthropic (along with NVIDIA's elevated valuation) are founded on their work.
When Attention is All You Need was published, the market as it stands didn't exist. It's like comparing the pre-Jordan NBA to post. Same game, different league.
I'd argue the top individual researchers figure into the overall AI spend. They are the people leading teams/labs and are a marketable asset in a number of ways. Extrapolate this further outward - why does Jony Ive deserve to be part of a $6B aquihire? Why does Mira Murati deserve to be leading a 5 month old company valued at $12B with only 50 employees? Neither contributed fundamental research leading to where we are today.
Seriously. The transformer coupled with tons of compute is why we got here. When that paper came out and people (AI researchers) saw the results many were confused or unconvinced. No one has any clue such an architecture would yield the results it has. AI systems has always been far more art than science and we still don’t even really know why it works. I feel like that idea being stumbled upon was sort of more luck than anything…