Comment by keeda

As other comments point out, the interesting part is that most of them somehow independently converged on the same take.

Consider the probabilities involved in this probabilistic database. Given the absolutely massive amount of data encompassing a veritable superset of all possible expressions of humanity -- encapsulated in various modalities of media including text, audio and video, across almost all major cultures and languages, spanning nations and decades -- somehow almost all these black box models (presumably) trained independently in almost as many separate labs converged on the same "controversial" take.

I mean, isn't it very interesting to contemplate what could have caused this? Is it something in the data itself, meaning it is something latent in all our cultural output? Is it something in the training, as you theorized, and is it in pre-training or RLHF? Is it something about the people training these models?

Even more interestingly, the opinion seems so self-evident (and well represented in various philosophies) that almost nobody here seems to disagree with it. So how is this controversial, really?

Is this truly what the latent space vector for "controversial", when multiplied across all the other vectors in their weights representing all the concepts gleaned from all that data above, produces?

Or are the models, sensitive to the prospect of being tested, diplomatically telling us something they know we might find mildly offensive yet begrudgingly accept as true?

Or are the models actually alien intelligences that really "think" this is controversial, while we're like "meh", and such discrepancies are the roots of what we call "misalignment"?

The questions are endless and fascinating.