Comment by minraws

"As a person you are not saving the server." Sure I believe you, I have complete faith in your works and in other humans, we have never had people sacrifice well being of other people for a little bit of money. That just doesn't happen.

Jokes aside, I feel like we attribute too much power of choice to these models, they aren't decision machines they are approximation machines based on what's the best outcome as per "average human understanding". This by no means is the "average human" output, but the direction is always towards the mean of what people feeding these models their thoughts consider acceptable.

This models aren't truth systems they are mean reversion systems, you can push them towards your mean with agentic harness and prompts, but they aren't decision machines.

For instance if just have to never give the model the choice to be able to do any harm, then it doesn't matter if it's system decides to do something wrong.