Comment by augunrik
3 days ago
Yeah! I also think that the ban was unintended.
I also think that’s a big clown show. People think that LLMs accidentally get good with security patterns. That is not the case, they included all of that in the training data. They could also have left out the knowledge.
I don't think you actually can avoid a subject very effectively. Some things might have to be derived from related examples and real time searches but ultimately a kid raised by helicopter parents is all the more dangerous on the day they find the censored materials.
> Yeah! I also think that the ban was unintended.
Still, it saves them the burden of actually providing the model that's too expensive for them to provide. They can now brag they have the best model at zero cost to them. And without people poking it and finding its limitations (the real ones, not externally imposed ones).
If they leave at all info about security patterns, you get an AI that knows how to code, but doesn't know what can make code insecure. That doesn't seem like a great idea.