Comment by ACCount39

It's very easy to tune an LLM for a "default" like "be sycophantic" or "be contrarian". It's easy to instill a semi-rigid "response template" like "agree with most of whatever the user says, but find at least one thing to nitpick about and contradict the user on it".

It's very, very hard to tune an LLM for a robust, durable "actually approach user queries with nuance and contradict the user where it's warranted".

Claude doesn't handle that so well, but ChatGPT is even worse. Talk to it enough and you'll feel the "default response template" in your bones.