There are zero laws against the design, manufacture, sale, or possession of cars with spinning blades and/or flamethrowers.
Such laws attach to the human operators of such cars, if they recklessly endanger others by driving them on shared, public roads.
But they're perfectly legal to own (and operate off-road). And, there's no technical barrier stopping people from illegally driving them on public roads: just fear of laws.
(There's entire subcultures of people who modify cars to do insanely dangerous things, on private race tracks. Perfectly legal).
If we were to apply Amodei's safety standard to cars—do they have the theoretical risk of killing over 50 people—well, cars empirically kill tens of thousands of people in the US annually, so we'd have to categorically ban them. That standard is bad-faith and absurd.
No. Half the point of LLM's is having them research topics on the internet as your agent. It's totally harmless in the usual (tool-based search) workflow.
(Would we also need a real ID to operate curl(1)?)
Regulations should attach only at the point humans willfully or recklessly cause their agents to commit crimes.
Yet we don't regulate computers so that they're rendered incapable of running software that can be used to hack into systems. Or said software, by the way. All we do is punish the people who use these tools to cause harm to others.
Technically no, but you can distribute software, especially for free, that happens to break copy protection for one or more valid reasons, according to the law.
And, more importantly, we certainly don't try to limit the capabilities of computing that might be used to do those things. Given how AIs are very general purpose tools, the fact that they can be used to do something potentially harmful is too narrow a threat.
Nobody is trying to push for cars that automatically stop themselves from running over someone either, which would make for a more comparable analogy than cars with flamethrowers.
That assumes AI is designed to be malicious, "the spinning blades". AI is not; the public version given to you is the sanded down one, in the same way that cars don't have spinning blades but can still be dangerous when misused.
If a person closes his eyes and hits the gas pedal on the car and runs over someone, the person is to blame, not the car. AI companies do the same. Your eyes may be closed but you're still operating/hosting the car/AI.
There being no third party checks is the point, and it's not bad.
Keep in mind that those open source/weight models are not as powerful as whatever closed source Claude/GPT version exists behind the scenes that hack companies. As per the Hugging Face hack - open weight GLM saved the day as closed source ChatGPT refused to help Hugging Face fix their systems. An example of my point that "sidestepping any third-party checks that could be present" IS good.
Part of point of open source/weight AI is that it allows what those model providers can't: uncensored models for whatever reason you may need it. You are responsible for being legal with it.
If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
Until then, I propose we use the 'unplug the server' rule. Can [insert bad thing] be prevented by unplugging the server? Then these things are fully in our control and no, we don't need to re-build hundreds of years of legal precedent from the ground up.
Liability in the event of harm is already something we have a pretty good handle on from living through the industrial revolution 150 years ago, a time when much more activity that companies did existed in the real world with real harms, as opposed to digital companies doing imagined "mental" harm to soft desk chair people who work at email jobs.
> If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
No, only metaphorical AI spinning blades and flamethrowers should be regulated. Actual AIs with spinning blades and flamethrowers will be subsidized and unleashed in your town.
There are zero laws against the design, manufacture, sale, or possession of cars with spinning blades and/or flamethrowers.
Such laws attach to the human operators of such cars, if they recklessly endanger others by driving them on shared, public roads.
But they're perfectly legal to own (and operate off-road). And, there's no technical barrier stopping people from illegally driving them on public roads: just fear of laws.
(There's entire subcultures of people who modify cars to do insanely dangerous things, on private race tracks. Perfectly legal).
If we were to apply Amodei's safety standard to cars—do they have the theoretical risk of killing over 50 people—well, cars empirically kill tens of thousands of people in the US annually, so we'd have to categorically ban them. That standard is bad-faith and absurd.
So we just make the regulation such that it kicks in on machines connected to the public web?
No. Half the point of LLM's is having them research topics on the internet as your agent. It's totally harmless in the usual (tool-based search) workflow.
(Would we also need a real ID to operate curl(1)?)
Regulations should attach only at the point humans willfully or recklessly cause their agents to commit crimes.
2 replies →
Yet we don't regulate computers so that they're rendered incapable of running software that can be used to hack into systems. Or said software, by the way. All we do is punish the people who use these tools to cause harm to others.
AI is much more comparable to that than to a car.
Is that right? You can't even sell software built to break copy protection. Can you legally sell software made to harm others?
You can still run someone over with a car. The analogy doesn't require absolute lack of capability to do harm.
Technically no, but you can distribute software, especially for free, that happens to break copy protection for one or more valid reasons, according to the law.
And, more importantly, we certainly don't try to limit the capabilities of computing that might be used to do those things. Given how AIs are very general purpose tools, the fact that they can be used to do something potentially harmful is too narrow a threat.
Nobody is trying to push for cars that automatically stop themselves from running over someone either, which would make for a more comparable analogy than cars with flamethrowers.
That assumes AI is designed to be malicious, "the spinning blades". AI is not; the public version given to you is the sanded down one, in the same way that cars don't have spinning blades but can still be dangerous when misused.
If a person closes his eyes and hits the gas pedal on the car and runs over someone, the person is to blame, not the car. AI companies do the same. Your eyes may be closed but you're still operating/hosting the car/AI.
There exist uncensored, unregulated models. Many of them are available for local use, sidestepping any third-party checks that could be present.
There being no third party checks is the point, and it's not bad.
Keep in mind that those open source/weight models are not as powerful as whatever closed source Claude/GPT version exists behind the scenes that hack companies. As per the Hugging Face hack - open weight GLM saved the day as closed source ChatGPT refused to help Hugging Face fix their systems. An example of my point that "sidestepping any third-party checks that could be present" IS good.
Part of point of open source/weight AI is that it allows what those model providers can't: uncensored models for whatever reason you may need it. You are responsible for being legal with it.
If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
Until then, I propose we use the 'unplug the server' rule. Can [insert bad thing] be prevented by unplugging the server? Then these things are fully in our control and no, we don't need to re-build hundreds of years of legal precedent from the ground up.
Liability in the event of harm is already something we have a pretty good handle on from living through the industrial revolution 150 years ago, a time when much more activity that companies did existed in the real world with real harms, as opposed to digital companies doing imagined "mental" harm to soft desk chair people who work at email jobs.
> If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
No, only metaphorical AI spinning blades and flamethrowers should be regulated. Actual AIs with spinning blades and flamethrowers will be subsidized and unleashed in your town.