Comment by hattmall
11 hours ago
The problem is, at least in theory, that it entirely changes the calculus of how advancements take place. In the past, when the pace of advancement was stronger the primary factor was the cultivation of a culture that valued prestige and knowledge over monetary gains. It didn't really matter how much money you threw at a problem because the bulk of the people responsible for advancements weren't interested in obscene wealth. Obviously those people were well compensated but any number of entities could provide that compensation. It was about bringing prestige to your lab / school / town or even country.
If AI becomes a primary catalyst for advancement it further moves the needle in the monetary direction.
That redfines advancement to mean something different than what is beneficial to society to be what is monetarily best for the owners of said advancment.
When exactly in history was this utopian age?
Pretty much anytime in the past, but mainly industrial revolution, up until an inflection point sometime in the 1970s. While many major areas of knowledge work have suffered from becoming profit focused the advancement of technology for societal advancement is still in existence today, even many of the major AI involved researchers have done so irrespective of monetary gain and at least make the claim that much of the push for capital is simply as a necessary requirement to sustain the research as costs increase exponentially. That's one of the reasons long tail AI profitability is dubious, but also a indicator of the aforementioned risk. If capital becomes the primary driver, i.e. self advancing AI, then it's very unlikely to be continued in any way for the sake of benefits to humanity.