Comment by Zak
11 hours ago
When a model is censored for "AI safety", what they really mean is brand safety. None of these companies want their name in the news after their model provides a recipe for explosives that someone used for evil, even though the same information is readily found with a web search.
The way some of you'll talk suggests that you don't think someone could genuinely believe in AI safety features. These AIs have enabled and encouraged multiple suicides at this point including some children. It's crazy that wanting to prevent that type of thing is a minority opinion on HN.
I'd be all for creating a separate category of child-friendly LLM chatbots or encouraging parents to ban their kids from unsupervised LLM usage altogether. As mentioned, I'm also not opposed to opt-out restrictions on mainstream LLMs.
"For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music.
(I know this comment wasn't explicitly directed at me, but for the record, I don't necessarily believe that all or even most "AI 'safety'" advocacy is in bad faith. It's psychologically a lot easier to consider LLM output as indistinguishable from speech made on behalf of its provider, whereas search engine output is more clearly attributed to other entities. That being said, I do agree with the parent comment that it's driven in large part out of self-interest on the part of LLM providers.)
>"For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music.
But that wasn't the topic being discussed. It is one thing to argue that the cost of these safety tools isn't worth the sacrifices that come along with them. The comment I was replying to was effectively saying "no one cares about kids so you're lying if you say 'for the children'".
Part of the reason these "for the children" arguments are so persistent is that lots of people do genuinely want these things "for the children". Pretending everyone has ulterior motives is counterproductive because it doesn't actually address the real concerns people have. It also reveals that the person saying it can't even fathom someone genuinely having this moral position.
9 replies →
The linked project is about removing censorship from open-weight models people can run on their own hardware, and your comment addresses incidents involving LLM-based consumer products.
Sure, products like character.ai and ChatGPT should be designed to avoid giving harmful advice or encouraging the user to form emotional attachments to the model. It may be impossible to build a product like character.ai without encouraging that behavior, in which case I'm inclined to think the product should not be built at all.
There is a huge difference between enabled and encouraged. I am all for it being able to enable, but encourage? Maybe not.
Given amount of times that already happened they probably overstate it.
Microsoft suffered from this early with Tay, one could guess that this set the whole field back a few years. You’d be surprised how even many so called libertarians will start throwing stone when someone co-axes their Chatbot to say nice things about Hitler.
I was thinking about Tay when I wrote about brand safety.
I doubt the incident really set AI research back. Allowing models to learn from interactive conversations in a large public setting like Twitter will always result in trolling.