Comment by oblio
2 days ago
> To start making models better again, AI coding companies need to invest in high-quality data, perhaps even paying experts to label AI-generated code.
Heh, there's only one problem with that. Training models is very expensive from a power/infrastructure/hardware perspective. Inference is not as expensive but it's still fairly expensive and needs sophisticated layers on top to make it cheaper (batching, caching, etc).
Guess in which cost category "high-quality data reviewed by experts" falls under.
I would hope the trillions of dollars sloshing around are used to pay people to make the core of the product better.
If you ask around Magnificent 7, a lot of the talk rhymes with: "we're converting Opex into Capex", translated: "we're getting rid of people to invest in data centers (to hopefully be able to get rid of even more people over time).
There are tons of articles online about this, here's one:
https://finance.yahoo.com/news/amazon-bets-ai-spending-capex...
They're all doing it, Microsoft, Google, Oracle, xAI, etc. Those nuclear power plants they want to build, that's precisely to power all the extra data centers.
If anything, everyone hopes to outsource data validation (the modern equivalent to bricklayers under debt slavery).