Comment by elephanlemon
19 hours ago
IMO we are currently in the ENIAC era of LLMs. Perhaps there will be a brief moment where things get worse, but long term the cost of these things will go way down.
19 hours ago
IMO we are currently in the ENIAC era of LLMs. Perhaps there will be a brief moment where things get worse, but long term the cost of these things will go way down.
I assume the "briefly gets worse" is when a buch of hyperscalers do a complete write-off of their entire AI investments, bankrupting several of them (which, in turn, bankrupts several large banks and most current venture capital firms)?
Cumulative AI capex will hit $2T this year. Cumulative opex is on the same order. Unless the models get real good (as in: can fully replace many engineers) right quick, nobody is even going to see interest getting paid on those investments. The only alternative is model access costing 5 figures per (replaced) seat.
But yes, once GPU racks can be had at auction for pennies on the dollar, inference of open source models might be an... OK low margin commodity business.
Cost will probably go down but nobody knows when or how. It might take 10 years for all we know as training costs have only been rising.
A huge difference is early computers were not subsidized. It took decades until most people could afford to own a computer at home.
Or we are in the early Netflix era where profit wasn’t as important as customer growth.