Comment by hedora
2 days ago
There's a finite amount of training data available, and multiple companies have downloaded all of it. Arguably, trying to get a monopoly on training data is equivalent to getting monopoly access to the open internet + books.
That ain't going to happen, and if it somehow did in the US, things like Anna's Archive + libgen will save the rest of the planet from our self-imposed dark ages.
As for those data centers: The cost of inference and training on the same hardware is dropping much faster than Moore's Law. That multiplies with hardware advances, since Moore's Law isn't quite done yet.
So, those datacenters are depreciating at 50% every 18 months (Moore's law) at the same time as performance/$ and performance/watt of the software is dropping at some crazy rate like 10x per year. So, token per unit of data center is increasing by something like > 20x every 18 months, and that'll probably happen at least one more time. If it does, the datacenters will be provisioned to produce 400x more tokens than they could have when they were sized.
Put another way, if the singularity is coming, the first things it will destroy are those data center build outs. That doesn't matter though: the data centers are being built on circular loans, so the money needed to complete them probably never existed.
Dark times ahead; hopefully Trump will be blamed, since all this corruption happened on his watch while he had control of the senate, house and (partially) the courts. The stock-goes-down graph will show it happening after the midterms, of course.
> The stock-goes-down graph will show it happening after the midterms, of course.
This, to me, is the most interesting question in all of this. I increasingly feel like we live in a 'post-truth' world with regards to money and the stock market in particular. Like even during the COVID lockdowns the market blinked and then was at record highs within 6 months. It was just lol and completely out of touch with the reality on the ground.
The fed, and government alike, seem to have gradually shifted their mandates from trying to provide to a stable playground to trying to create a playground where important kids are never allowed to fall down. And when your existence is tied to trillions of dollars of the economy, you become the important kid.
Come to think of it this is also probably being exasperated by the rise of money management firms controlling tens of trillions of dollars. When the market valuation is technically just based on sentiment and you control that much money in a self-interested way, you're going to get weird outcomes.
Chancellor On The Brink