Comment by frereubu
9 hours ago
Why aren't these agents set up to do what a security researcher should do - responsible disclosure, ideally to a specific person in its company to handle, or I suppose potentially directly to the organisation itself e.g. if they have a security.txt file on their website? I really hope legal precedent is quickly established that holds companies responsible for the actions of their agents.
Because they’re misaligned and cannot be just "set to do" <a reasonable thing>?
I know it sounds a bit like "don't make mistakes", but surely this could be part of the core instructions? Recognising categories of sensitive data like medical data and having some kind of check-in with whoever has asked it to do something?
They don’t follow even the core instructions reliably enough.