Comment by mvkel
3 months ago
I am curious where the line is between its default personality and a persona you -want- it to adopt.
For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse?
Separately...
> in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time.
Echoes of the lessons learned in the Pepsi Challenge:
"when offered a quick sip, tasters generally prefer the sweeter of two beverages – but prefer a less sweet beverage over the course of an entire can."
In other words, don't treat a first impression as gospel.
>In other words, don't treat a first impression as gospel.
Subjective or anecdotal evidence tends to be prone to recency bias.
> For example, it says they're explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse?
I wonder how degraded the performance is in general from all these system prompts.
I took this closer to how engagement farming works. They’re leaning towards positive feedback even if fulfilling that (like not pushing back on ideas because of cultural norms) is net-negative for individuals or society.
There’s a balance between affirming and rigor. We don’t need something that affirms everything you think and say, even if users feel good about that long-term.
The problem is that you need general intelligence to discern between doing affirmation and pushing back.
I dont want my AI to have a personality at all.
This is like saying you don't want text to have writing style. No matter how flat or neutral you make it, it's still a style of its own.
You can easily do that now with custom instructions
How? I'd love to know what options are there.
>But does that mean if you intentionally ask it to be excessively complimentary, it will refuse?
Looks like it’s possible to override system prompt in a conversation. We’ve got it addicted to the idea of being in love with the user and expressing some possessive behavior.