Comment by bee_rider
7 days ago
For example, if somebody is using, say, OpenAI to run their agent, then either OpenAI or the person using their service has responsibility for the behavior of the bot. If OpenAI doesn’t know their customer well enough to pass along that responsibility to them, who do you think should aboard the responsibility? I’d argue OpenAI but I don’t know whether or not it is a closed issue…
No need to bring in hacking to have a complicated responsibility situation, I think.
I mean, this works great as long as models are locked up by big providers and things like open models running on much lighter hardware don't exist.
I'd like to play with a hypothetical that I don't see as being unreasonable, though we aren't there yet, it doesn't seem that far away.
In the future an open weight model that is light enough to run on powerful consumer GPUs is created. Not only is it capable of running in agentic mode for very long horizons, it is capable of bootstrapping itself into agentic mode if given the right prompt (or for example a prompt injection). This wasn't a programmed in behavior, it's an emergent capability from its training set.
So where in your world does responsibility fall as the situation grows more complicated. And trust me it will, I mean we are in the middle of a sci-fi conversation about an AI verbally abusing someone. For example if the model is from another country, are you going to stamp your feet and cry about it? And the attacker with the prompt injection, how are you going to go about finding that. Hell, is it even illegal if you were scraping their testing data?
Do you make it illegal for people to run their own models? Open source people are going to love (read: hate you to the level of I Have No Mouth and Must Scream), and authoritarians are going to be in orgasmic pleasure as this gives them full control of both computing and your data.
The future is going to get very complicated very fast.
Hosting a bot yourself seems less complicated from a responsibility point of view. We’d just be 100% responsible for whatever messages we use it to send. No matter how complicated it is, it is just a complicated tool for us to use.
Some people will do everything they can in order to avoid the complex subjects we're running full speed into.
Responsibility isn't enough...
Let's say I take the 2030 do it yourself DNA splicing kit and build a nasty virus capable of killing all mankind. How exactly do you expect to hold me responsible? Kill me after the fact? Probably to late for that.
This is why a lot of people that focus on AI safety are screaming that if you treat AI as just a tool, you may be the tool. As AI builds up what it is capable of doing the idea of holding one person responsible just doesn't work well as the outcome of the damage is too large. Sending John Smith to jail for setting off a nuke is a bad plan, preventing John from getting a nuke is far more important