Comment by cyanydeez

Alignment isnt just POV problem.

Its that LLMs are not deterministic. If you want it to not talk about nuclear weapons, you have to teach it all about them otherwise if has nothing to align against.

Then its trivial to invert its alignment and it has all the nucleat data.

Nothing abouT LLM alignment makes sense.