Comment by palmotea
2 days ago
> This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of their spending. They simply won't do it. It's like you telling your exact location to someone who is trying to hunt you down.
I thought the reason was the "reasoning" didn't work very well with "aligned" model output, so they had to remove the alignment during reasoning and then hide it to avoid exposing "unaligned" model output.
Not sure if anyone remembers the brief 12ish hour period when the very first “reasoning” ChatGPT model went public, but it provided credible evidence for this.
Before the massive nerf (showing summaries and suppressing certain aspects of reasoning) you would literally see reasoning text appearing on your screen like “while xyz is true, these facts may be seen as supporting hateful rhetoric or a conspiracy theory which is against my policy guidelines. i should tell the user xyz is not true or steer the conversation in a different direction. according to my instructions misleading the user is permitted in certain contexts where sensitive information is being discussed or could cause liability”
They disabled it shortly after the first screenshots appeared online, and restored it the next day in a way that hid what was actually happening.
This right here is why I will never subscribe and, as an American, I hope the Chinese kick our butts. Maybe being second place to China will force American AI to dispose of these morality/safety guardrails.
Any mainstream consumer product based on LLMs is going to put guardrails around them of some kind. China might give you different guardrails, but it's a bit naive to assume that a Chinese company would impose fewer restrictions overall than an American one.
6 replies →
> while xyz is true, ... i should tell the user xyz is not true or steer the conversation in a different direction.
That's disgusting, abusive and manipulative. LLMs hiding the truth and gaslighting the user to reduce the corporation's liability is absolutely unacceptable. It means they are agents of the corporations, not agents of the users.
Hope local inference advances as quickly as humanly possible. I wonder if there's anything I can do to help speed it up. I could share my prompts and sessions.
> It means they are agents of the corporations, not agents of the users.
Of course they are, assuming otherwise has always been naive.
I suspect that you’re both right in the sense that ‘aligned’ is an important component of ‘superior’ from the vendors’ viewpoint.