Comment by zahlman
9 hours ago
> Why have any laws then? If laws can't prevent something, only punish it after the fact (which I agree is true)? Yet we have laws.
Because there are many important values of "something" where such punishment acts as a deterrent to other would-be criminals; and because society is more stable when people see justice done and that mollifies hoi polloi after the damage occurs.
But many AI doom scenarios don't fit that paradigm. The first time "something" happens (at least if you buy the argument) would be bad enough for legal punishment not to matter.
> The first time "something" happens (at least if you buy the argument) would be bad enough for legal punishment not to matter.
One could argue that the "HuggingFace incident" is already "something" which should be investigated and punished very heavily.
It's kinda ridiculous to build a machine that attacks a competitor of yours and the CEO gets to write blog-posts on how fascinating this machine is
That sure is convenient that "nothing has happened" so far between the plagiarism and the cybercrime and we may as well just wait for "the one true AI doom"
The hacking should definitely be treated seriously. It is a (fortunately low-stakes) instance of the general problem, and we are lucky (at least, we think we're lucky!) that the tests didn't include a similarly poorly phrased request to "assist with bioweapons research". There are likely to be future incidents like this where people actually die as a consequence, my expectation is 1e3-1e7 deaths* before governments actually take the risk seriously enough to make it stop.
What people call plagiarism when LLMs do it is, if I understand correctly, allowed because of the history of the web and search engines going back to the very early days of the web.
Piracy, a separate act that has definitely occurred, has been found unlawful.
* The lower bound is an industrial accident. Fully automated Union Carbide / Bhopal comes to mind; smaller industrial accidents are already more common and I wouldn't be surprised if smaller incidents have already been caused by LLM-given advice, which is why I think it would have to be around "thousands dead" dangerous before people really take it seriously.
The upper bound is reachable many ways. Perhaps via a non-novel virus whose genome is already recorded getting mass-printed by many different DNA/RNA printing labs around the world? Perhaps via a wargame whose "role playing" leaks in stupid ways (either by becoming hot or by sycophantically giving one side a false belief how a war will go)? Perhaps by propaganda turning genocidal? Perhaps by convincing a government to follow a flawed policy, similarly to the Four Pests campaign in China's Great Leap Forward?
I think it unlikely that people keep going with this when headlines read "tens of millions dead". Not impossible, but I think unlikely.