Geopolitics gets in the way. It is a race between America and China.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
It doesn't have to be a race, that's just the way we have been treating it so far. There have been examples in the past of civilization ending technology that has been redirected from being a death race (chemical, biological, and nuclear arms in particular.)
Not sure nuclear arms is a great example. There are more countries with nuclear weapons now than in 1970. India, Pakistan, and North Korea have all joined the nuclear club since then. The SALT and START treaties are no longer in effect.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
even if both sides agree to a pause china will annihilate the us in the robotics race (for which there will be no slowdown). software/llms is america's only edge
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
if it's a political issue then this is just Sam telling people to do one thing while still riding his SVPs and other leads hard on roadmaps, 'boldness', and delivery metrics
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly
Geopolitics gets in the way. It is a race between America and China.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
It doesn't have to be a race, that's just the way we have been treating it so far. There have been examples in the past of civilization ending technology that has been redirected from being a death race (chemical, biological, and nuclear arms in particular.)
China doesn't regard AI as a death race. It regards it as a way to ensure its own global stature and benefit its own population.
4 replies →
Not sure nuclear arms is a great example. There are more countries with nuclear weapons now than in 1970. India, Pakistan, and North Korea have all joined the nuclear club since then. The SALT and START treaties are no longer in effect.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
even if both sides agree to a pause china will annihilate the us in the robotics race (for which there will be no slowdown). software/llms is america's only edge
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
The AI race is caused and being sustained by American tech companies. China is an excuse to keep it going.
if it's a political issue then this is just Sam telling people to do one thing while still riding his SVPs and other leads hard on roadmaps, 'boldness', and delivery metrics
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly