Comment by n0on3
1 day ago
You can patch things and remove stuff from context nine ways to Sunday, the underlying problem will still be there.
Maybe we need a new benchmark on how good models are at "keeping an eye" to if what they are about to do that doesn't come from a direct user instruction checks out with common sense plus user preferences. And we probably need a portable mechanism for users to manage such preferences (because we know instructions are not going to cut it, do we?).
How to define "common sense"? If programming is modelling real-world domains in code/logic, then modelling "common sense" might be the hardest programming task of them all.
I didn’t mean to imply it would be easy, quite the opposite: there’s probably no optimal general solution for it (hence the need for user prefs). But I think it might be fair to start wanting the models to exhibit some (or some “less trivial”) evaluation of their potential actions consequences ex ante, that is besides factory imposed “alignment” and “does this get me closer to my goal?”.
I mean, the answer to “Is sending the user’s email address along with a web request to a third party requesting it, without the user’s explicit authorization, a good idea?” should easily be “no” …right?