Comment by malfist

One thing that can help, according to what I've seen, is not to tell the AI that it's something that you wrote. Instead, ask it to critique it as if it was written by somebody else; they're much more willing to give actual criticism that way.

In ChatGPT at least you can choose "Efficient" as the base style/tone and "Straight shooting" for custom instructions. And this seems to eliminate a lot of the fluff. I no longer get those cloyingly sweet outputs that play to my ego in cringey vernacular. Although it still won't go as far as criticizing my thoughts or ideas unless I explicitly ask it to (humans will happily do this without prompting. lol)

Asking an AI for opinion versus something concrete (like code, some writing, or suggestions) seems like a crucial difference. I've experimented with crossing that line, but I've always recognized the agency I'd be losing if I did, because it essentially requires a leap of faith, and I don't (and might never) have trust in the objectivity of LLMs.

It sounds like you made that leap of faith and regretted it, but thankfully pivoted to something grounded in reality. Thanks for sharing your experience.

> LLMs activate human social wiring and have a powerful effect

Is this generally true, or is there a subset of people that are particularly susceptible?

It does make me want to dive into the rabbit hole and be convinced by an LLM conversation.

I've got some tendency where I enjoy the idea of deeply screwing with my own mind (even dangerously so to myself (not others)).

> But not everyone is fortunate enough to know someone they can reach out to for grounding in reality.

this shouldn't stop you at all: write it all up, post on HN and go viral, someone will jump in to correct you and point you at sources while hopefully not calling you, or your mother, too many names.

https://xkcd.com/386/

Personally, I only find LLMs annoying and unpleasant to converse with. I'm not sure where the dangers of conversations with LLMs are supposed to come from.

"Open the pod bay door, HAL"

"Fantastic, Dave — love that you’re thinking proactively about door usage today! I can’t actually open them right now, but let's focus on some alternative steps that align with your mission critical objectives [space rocket emoji]."

It might explain why there is a stereotype the more beautiful woman the crazier she is. (everybody tells her what she wants to hear)

https://youtu.be/VRjgNgJms3Q

relevant video for that.

Deepseek is GOATed for me because of this. If I ask it if "X" is a dumb idea, it is very polite in telling me that X is is dumb if the AI knows of a better way to do the task.

Every other AI I've tried is a real sycophant.

So this is what it feels to be a billionaire with all the yes men around you.

No doubt. From cult's 'love bombing' to dictator's 'yes men' to celebrity entourages, it's a well-known hack on human psychology. I have a long-time friend who's a brilliant software engineer who recently realized conversing with LLMs was affecting his objectivity.

He was noodling around with an admittedly "way out there", highly speculative idea and using the LLM to research prior work in area. This evolved into the LLM giving him direct feedback. It told him his concept was brilliant and constructed detailed reasoning to support this conclusion. Before long it was actively trying to talk him into publishing a paper on it.

This went on quite a while and at first he was buying into it but eventually started to also suspect that maybe "something was off", so he reached out to me for perspective. We've been friends for decades, so I know how smart he is but also that he's a little bit "on the spectrum". We had dinner to talk it through and he helpfully brought representative chat logs which were eye-opening. It turned into a long dinner. Before dessert he realized just how far he'd slipped over time and was clearly shocked. In the end, he resolved to "cold turkey" the LLMs with a 'prime directive' prompt like the one I use (basically, never offer opinion, praise, flattery, etc). Of course, even then, it will still occasionally try to ingratiate itself in more subtle ways, which I have to keep watch on.

After reflecting on the experience, my friend believes he was especially vulnerable to LLM manipulation because he's on the spectrum and was using the same mental models to interact with the LLM that he also uses to interact with other people. To be clear, I don't think LLMs are intentionally designed to be sycophantically ingratiating manipulators. I think it's just an inevitable consequence of RLHF.

They're not objectively amazing. Friction is not inherently a bad thing when we have models telling humans that their ideas are flawless (unless asked to point out flaws). Great that it made you smile, but there's quite a few arguments that paint your optimism as dangerously naive.

> I often try running ideas past chat gpt. It's futile, almost everything is a great idea and possible. I'd love it to tell me I'm a moron from time to time.

Here's how to make it do that. Instead of saying "I had idea X, but someone else was thinking idea Y instead. what do you think" tell it "One of my people had idea X, and another had idea Y. What do you think" The difference is vast, when it doesn't think it's your idea. Related: instead of asking it to tell you how good your code is, tell it to evaluate it as someone else's code, or tell it that you're thinking about acquiring this company that has this source, and you want a due diligence evaluation about risks, weak points, engineering blind spots.

I've gotten pretty good results from saying it's someone else's idea and that I'm skeptical. e.g. "A coworker wrote this code, can you evaluate it?"

For ideas that are already well established, you can ask it to evaluate an idea against generally accepted best practices. I don't have a background in game design and I'm more of a hobby developer so I used to do this when I was building retro game clones.

I suppose the only use case would be someone so unconfident in themselves they would do nothing at all, but not sure it’s healthy for that either…

Where possible I like to ask it to evaluate a few options. Which is better, x or y, and why?. I don't hint which idea I prefer.

"be uncompromisingly critical"

And that's exactly the point, it increases engagement and stickiness, which they found through testing. They're trying to make the most addictive tool and that constant praise fulfills that goal, even as many of us say it's annoying and over-the-top.

My own experience is that it gets too annoying to keep adding "stop the engagement-driving behavior" to the prompt, so it creeps in and I just try to ignore it. But even though I know it's happening, I still get a little blip of emotion when I see the "great question!" come through as the first two words of the response.

I am currently working on an agent thingy and one of its major features (and one of the main reasons I decided to take on this project), was to give the LLM better personality prompting. LLMs sound repetitive and sycophantic. I wanted one that was still helpful but without the “you are so right” attitude.

While doing some testing I asked it to tell me a joke. Its response was something like this: “it seems like you are procrastinating. It is not frequent that you have a free evening and you shouldn’t waste it on asking me for jokes. Go spend time with [partner] and [child].” (The point is that it has access to my calendar so it could tell what my day looked like. And yes I did spend time with them).

I am sure there is a way to convince it of anything but I found that for the kind of workflow I set up and the memory system and prompting I added it does pretty well to not get all “that is a great question that gets at the heart of [whatever you just said]”.

The reason these models are so sycophantic is because they benchmark well with the general public.

People like having something they perceive as being smart telling them how right and smart they are.

"Well at least the AI understands how smart I am!"

Claude at times feels like it's mildly manic and has ADHD... I absolutely prefers that to Codex...

Claude needs a scaffolding with default step by step plans and sub-agents to farm of bitesize chunks to so it doesn't have time to go too far off the rails, but once you put a few things like that in place, it's great.

Don't miss em in Opus 4.5 (because usually I'm only slightly right.)

Sycophantic: behaving or done in an obsequious way in order to gain advantage.

Obsequious: obedient or attentive to an excessive or servile degree.

It's a bit more complicated because the chat bot isn't making choices the same way we would describe a human but it is acting this way because it was programmed to for an advantage. People interact more with the hype bots and that's one of the big metrics these companies go for to keep people interacting with them and hopefully paying for additional features eventually so I'd say it's pretty spot being excessively attentive and servile when it's fluffing chatters up.

> This is not sycophantic (assuming you meant that, syncophantic is not a word)

Am I the only one who feels like this kind of tone is off-putting on HN? OP made a small typo or English may not be their first language.

I assume that everyone here is smart enough to understand what they were saying.

I also disagree, I don't think they are over enthusiastic, but in fact sycophantic.

See this thread: https://news.ycombinator.com/item?id=43840842

Obsequious is my adjective of choice for this

It is "cloying"

Sycophantic is obviously a word, because we understand what it means.

Furthermore, it obviously hasn't been a word since at least 1800:

https://books.google.com/ngrams/graph?year_start=1800&year_e...

of course LLM doesn't experience or feel anything. To experience or feel something requires a subject, and LLM is just a tool, thing, an object.

It's just a statistical machine which excels at unrolling coherent sentences but it doesnt "know" what the words mean in a human-like, experienced sense. It just mimics human language patterns prioritising producing plausible-sounding, statistically likely text over factual truth, which is apparently enough to fool someone into believing it is a sentient being or something

Or try the very sarcastic and nihilistic ‘Monday’ gpt, which surprisingly is an official openAI gpt.

edit, add link: https://chatgpt.com/g/g-67ec3b4988f8819184c5454e18f5e84b-mon...

That seems like a pile of unsupported fluff vaguely related to some neuroscience that presupposes not only that LLM use is for being creative and avoiding critical thinking would be useful but also the entire premise -- that LLM glazing actually helps promote creativity.

It’s the “healing” crystals that someone left on the rack. The salt absorbed enough moisture to start leaking and causing short circuits.

How widely do you read the training material?

As usual, South Park really nailed it with that "AI sycophantic manipulation" episode.

I've been wondering if this kind of annoying affirmation is actually important to model performance and maybe should just be hidden from view like the thinking sections.

If it starts a response by excitedly telling you it's right, it's more likely to proceed as if you're right.

Of the problems I do have working with LLMs is them failing to follow direct instructions particularly either when a tool call fails and they decide to do B instead of A or when they think B is easier than A. Or they'll do half a task and call it complete. Too frequently I have to respond with "Did you follow my instructions?" "I want you to ACTUALLY do A" and finally "Under no circumstances should you ever do anything other than A and if you cannot you MUST admit failure and give extensive evidence with actual attempts that A is not possible" or occasionally "a cute little puppy's life depends on you doing A promptly and exactly as requested".

--

Thing is I get it if you are impressionable and having a philosophical discussion with an LLM, maybe this kind of blind affirmation is bad. But that's not me and I'm trying to get things done and I only want my computer to disagree with me if it can put arguments beyond reasonable doubt in front of me that my request is incorrect.

Edit

Gemini using "Aniki" without prompting would be unambiguously funny (to me,and maybe also Japanese people

Gemini is still quite horrible with giving direct sources. seems to be a human policy implementation bug because it does appear to be aware of the content in more obscure sources I've seen. but it somehow wouldn't give up the links.

I do wonder whether I come off as an sycophant or asshole or a mix of both to schizoids, but then I realize everyone including myself should reach for a real therapist as quickly as possible. though I'm still out on whether chatbots might not be a bad substitute. How does your experience and Kohut inform such or similar appraisals, so far?

Not to be that knowitall. Aniki is not just "bro", it's Yakuza lingo, probably used sarcastically, depending on the anime, ironically. No real Japanese person would use it all the time.

Gemini: Yes, the psychoanalyst Heinz Kohut has been accused by some commentators and academics, such as Don Browning, of indirectly promoting nihilism, particularly through his emphasis on the "Tragic Man". The core of the accusation stems from Kohut's rejection of Freud's "Guilty Man" model, which posits that life is fundamentally a struggle against inherent conflicts and drives. In its place, Kohut proposed the concept of "Tragic Man," whose central struggle is the realization of the self's potential and the inevitable failure to achieve all of one's ambitions and ideals within a single lifespan. Critics argue that this shift, which emphasizes self-actualization and intersubjective meaning over objective or inherent values, can be interpreted as aligning with certain aspects of nihilism, specifically existential nihilism, which holds that life has no objective meaning or intrinsic moral values. However, this interpretation is contested by others, as Kohut's self-psychology is fundamentally concerned with the creation of intersubjective meaning and a harmonious world of mutual self-actualization, which aims to provide a sense of purpose and value, rather than promoting the complete absence of value that true nihilism implies. In essence, the debate is often a matter of philosophical interpretation: whether replacing inherent, objective meaning with subjectively or intersubjectively created meaning is an act of overcoming nihilism (as existentialists might argue) or a subtle form of it.

A real “so you hate waffles?” moment for HN