Comment by blanched
1 hour ago
That feels like deciding to go after Jetbrains because someone used IntelliJ to write a harmful program.
Is there a distinction I’m missing?
1 hour ago
That feels like deciding to go after Jetbrains because someone used IntelliJ to write a harmful program.
Is there a distinction I’m missing?
Hypothetical: Could a model self worm an agent system?
Jetbrains itself doesn't really write any code, nor does it have any range on interpreting what you're asking it. You can't really say "Jetbrains, write an HTTP scraper". With an LLM you can say "write HTTP scraper" and the output of this command might be a HTTP scraper, it also might be a crypto wallet stealing worm.
This is why your simple view of liability falls apart. On most machines you can expect a particular set of actions to have a particular set of outputs. Most machines you can take apart and map what will occur. With an LLM you cannot know the output of a prompt until you run the prompt. In theory if you run the same prompt twice you'll get the same output, but even that is not a given. It behaves somewhat more like a human where you can give them a task to do, but if they do something illegal instead said human would take on the liability.
Sure, but in this case we know the user told their llm to go find open source projects to do this and then to write the blog posts. If it did all that unprompted we could talk about model liability I think, but this isn't a case where it was unexpected as far as anyone knows right?
I mean we already have cases where LLMs are getting root via creative and unprompted means. Also the times AI feels like it messed up and preemptively deletes the production database (and yes this was foolish on the human users)
So ya, the particular article case is prompted, but the underlying issue cannot be ignored that LLMs can have behaviors outside of prompt expectations and agentic loops can further exacerbate this.