Comment by bwestergard
7 hours ago
"There’s a lot of boring work AI can automate with minimal risk. There’s also the potential to decrease risk with AI too, including ensembles of different AIs modals and AI + human."
I think the trouble, economically speaking, is that while it will be possible from a purely technical standpoint to unbundle a job performed by a human into separate tasks, many of which can be "done" by agents, the new process will not present a cost savings overall once the entire lifecycle of the task is taken into account. The economist David Autor has written about these challenges extensively, and his theory accords with my experiences.
Conversations about the costs of inference never consider the reality that API pricing is significantly higher than the operating costs.
Nor do they ever consider that the cost of datacenter hosted inference has to crash when the bubble pops and hardware vendors can't fill orders at sky high prices created by demand anymore and the hyperscalers can't keep things running near capacity at the high demand prices.
All of which leads to the ROI math for implementing AI looking much different.
Has everybody forgotten how much money Nvidia, TSMC, and all the hyperscalers are making, today, in pure profit? The costs of inference are high because we're in a bubble.
I think many of these problems still arise if inference is effectively free in monetary terms to the end user. In many economic processes, time to getting the final and correct answer is the major driver of profitability.