Comment by andrewla

This is part of the reason why alignment is a kind of poorly defined term, and it isn't just a property of the model. It's instead a property of the harness and the context.

A model (like a human) should be able to play a video game where decisions are made that in the real world would be terrible; if we remove that ability we intrinsically limit model capability. But in a Last Starfighter / Enders Game / JOSHUA scenario this could result in behavior in the real world that appears unaligned.