Comment by voidspark

Even with the sycophantic system prompt, there is a limit to how far that can influence ChatGPT. I don't believe that it would have encouraged them to become violent or whatever. There are trillions of weights that cannot be overridden.

You can test this by setting up a ridiculous system instruction (the user is always right, no matter what) and seeing how far you can push it.

Have you actually seen those chats?

If your friend is lying to ChatGPT how could it possibly know they are lying?