Comment by alecst
16 hours ago
Most feel-good pieces on AI make the same mistake, which is looking at where the ball is right now and not where it's going. If progress were to halt today, this article would be relevant and I'd upvote it.
But since the richest and least-trustworthy people on earth are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.
> Most feel-good pieces on AI make the same mistake, which is looking at where the ball is and not where it's going.
Humans have a bad track record predicting where a technology is going.
I do not know where AI heads in the next 5-25 years (I have my hypotheses, but they are likely off, too), but I see a useful tool that allows many people to experiment with and try things that they could not before and I do not see why should I subscribe to the doomer view.
As far as the moral character of the leaders: that's common, too. Many luminaries were, based on recollections, seen as pretty bad human beings and that didn't run off on their inventions. My 2c.
Being a doomer sucks, man. But I see a lot of issues with your way of thinking, which I was basically getting at in the post you replied to. Let me explain:
Today it's a useful tool. But in not too long--a few years, perhaps--this tool will be able to outsmart you. If you think this is definitely not going to happen, then okay--we see the progress in LLMs differently. I see machines which, in a matter of three years, went from being able to do basic proofs, to math Olympiad problems, to century-old Millennium problems. I see no reason why progress will slow down.
Any "tool" which is that much smarter than you is no tool, any more than you are the "tool" of your pet dog.
I understand wanting to feel better and be optimistic about the future. I'm unfortunately not, and that makes a lot of people feel bad. Again, I'm sorry! I don't want to be a downer. But it's important to me that people take these risks seriously and don't minimize them with expressions like "we still have paper" and "it's a useful tool."
> Any "tool" which is that much smarter than you is no tool
I don't think this is accurate. Chainsaw is way more powerful than I am, yet it is definitely a tool. Computer is a tool too.
> I understand wanting to feel better and be optimistic about the future.
Historically, people who were pessimistic about the future, were very wrong. And this goes back centuries, if not millennia.
1 reply →
> But I see a lot of issues with your way of thinking, which I was basically getting at in the post you replied to.
Of course. And I am not claiming that my view is the view or even a particularly good view. It is just my personal worldview. But I think that with ups and downs, over generations and centuries, the world is on an uptrend. My parents were being driven, with their families, from their homes as kids; one of them lost an eye (as a kid) from a war; both lived through years of hunger. My grandparents saw real crap, two lived through a famine and saw many die from hunger.
But they all lived through those disruptions, worked and tried to build something better (I had to probe that information out of them; they did not even mention those things to us). I think the AI disruptions will not be nearly as painful and we will live through them just fine. Again, just a personal perspective.
> I see machines which, in a matter of three years, went from being able to do basic proofs, to math Olympiad problems, to century-old Millennium problems. I see no reason why progress will slow down.
I think what we see is LLMs picking the low hanging fruit. There is a lot of it, but I do not think we can extrapolate it. Some things will be particularly easy for LLMs to exceed humans on (like a steam engine being better in driving heavy machinery), some possible, many very hard. And humans will find new things to explore and build. We will see in 25 years (I am old and might not be alive then, but maybe I will see enough in 15).
As a side note: I do not know why you are being downvoted. I do not agree with your view, but I think it is intelligent and well written. I hope folks stop downvoting anything they disagree with and go back to downvoting on tone, not alignment.
3 replies →
> Humans have a bad track record predicting where a technology is going.
Specifically, they tend to overestimate the short term and underestimate the long term, or so it’s said.
(Amara’s law: https://en.wikipedia.org/wiki/Roy_Amara )
All I hear when I read the “but it’s just a useful tool” takes is “hey it’s fine if we use LLMs to kill people and take away their dignity and ability to make a living because it generates documents for me”.
Wealth is already concentrated. They’re just coming for the top 10% that affect the software devs now since they’ve taken everything else. You talked to anyone with an “unskilled” job in the last few years?
> Wealth is already concentrated.
Which is bad, and higher concentration makes the situation worse.
The thing that strikes me about the 'paperless office' as it was hyped isn't how much paper use was reduced, but what was enabled or sped up by computer databases or the internet to work with a head office. Consider how restricted most mail order companies would be compared to the modern market if they could only communicate with their customers, suppliers, logistics and other partners via phone/fax/physical mail, and they either had one central records department or the difference in efficiencies handling a network of local branches/franchises.
That leads to pondering if we're actually seeing a similar shift today with AI.
They're going to do all those things with or without AI. We need to fight the end goals at a social level rather than the temporary means
> They're going to do all those things with or without AI.
Yes, but AI gets them closer to those things faster.
> We need to fight the end goals at a social level rather than the temporary means
Again, yes, we need to fight the end goals, but we also have to stop the harm caused by the means. If there’s an arsonist going around burning people’s homes we do have to catch them but we also need to put out the fire on the houses burning right now (imperfect analogy, sure, as they all are).
> are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.
While two out of those three risks do not require anything "ultraintelligent", for killing most jobs it's worth remembering that their track is an appropriation of intelligence, and the speedups happen where it's easiest to measure more than where the tracks correspond to economic value.
Unfortunately, this has the opposite effect for safety, and can still do a lot of wealth concentration even if it doesn't end all jobs.
> killing most jobs
I can think of a dozen jobs off the top of my head that aren’t even remotely at risk due to LLM-style AI. And if your reply is, it’s only a matter of time until humanoid robots replace all human jobs, you’re very divorced from reality. In hundreds of years, maybe. Not in your lifetime.
This tech will already have a massive effect on society, purely from being able to generate text and software code at will. What is the constant need to exaggerate this into a Sci-fi story?
Also if that many people were put out of a job, to the level that their basic needs were no longer met, there would be mass unrest/revolution and all of the robot production and data centers would be burned to the ground. See: French Revolution, Russian Revolution, etc.
And it would almost certainly happen before it got to the point of "they have enough robots to stop the masses". And probably we'd hit 3.5% rule[1] non-violent government regulation of AI before any of that.
The real risk of AI to me is that we're too heavily invested in it and when the bubble pops it could take a lot of institutions with it. And governments are not in a great position to bail economies out again.
[1] https://en.wikipedia.org/wiki/3.5%2525_rule
[dead]
You live in a sci-fi world, we are that sci-fi story. Nothing we're talking about is outside the world of scientific possibility.
One thing you are really missing is a small amount of job displacement is going to create a huge deflationary amount in wages. At the end of the day you don't need that many plumbers and electricians. There was a reason a lot of people moved to office and tech jobs starting in the 80s. Trades were a dead end at the time. And the same thing will happen again as labor shifts.
This is making the same mistake: looking at where AI is now and not at the rate of progress it’s made. If you have a thinking machine that is more capable than every human on Earth and thinks 100x faster, how long do you think it will take for it to design humanoid robots.
I don’t think designing is very difficult. It’s actually having the resources and economic capacity to make millions of them, enough to “replace all jobs.”
It’s also a fallacy to assume that current progress implies future progress at the same rate. This is very basic technological history.
Again, a sci-fi fantasy for at least the next century.
2 replies →
>But since the richest and least trustworthy people on earth are on track to
They are on track to kill themselves with their decisions.
That's no consolation to all the people they're going to kill with them.
The important thing is a lot of them will die really.
> Most feel-good pieces on AI make the same mistake, which is looking at where the ball is and not where it's going.
Or imagining it’s going to the most utopian place possible, completely ignoring all the history which indicates the contrary.
Extinction will come not with a bang but with a whimper. The boundary of the burgeoning poor — already at two nines — will expand to include pretty much everyone, and that underclass will dominate society to the detriment of everything in the commons: culture, civility, affordable food, education, thinking.
Western culture devolves into poor, corrupt, violent, petty criminality. Welcome to Mega Cairo.