Comment by k1t
3 days ago
You are definitely not alone.
https://www.wsj.com/tech/ai/chatgpt-chatbot-psychology-manic...
Irwin, a 30-year-old man on the autism spectrum who had no previous diagnoses of mental illness, had asked ChatGPT to find flaws with his amateur theory on faster-than-light travel. He became convinced he had made a stunning scientific breakthrough. When Irwin questioned the chatbot’s validation of his ideas, the bot encouraged him, telling him his theory was sound. And when Irwin showed signs of psychological distress, ChatGPT assured him he was fine.
He wasn’t.
That’s why I always use a system prompt and tell it to be critical and call me out when I’m talking bullshit. Sometimes for easier queries it’s a bit annoying when I don’t actually need a “critical part” in my answers but often it helps me stop earlier when I’m following an idea that’s not that’s not as good as I thought it would be.