Comment by ceejayoz
2 hours ago
> It's pretty easy for a reasonable person to determine when to call 911…
Unfortunately, it's fairly tough for someone to determine if they are themselves a reasonable person.
A model that only ever says yes is a cognitohazard, IMO. Like being a billionaire without the money to insulate you from the consequences.
Saying yes is not equal to "provides the requested information." If you use models for affirmation instead of information, this is a more reasonable concern.
"I don't get to hear that it's not a doctor and I should call 911."
Sometimes, the requested information should be accompanied with a note like that.