Comment by dluan
3 hours ago
It'd be interesting to use this technique to create a running tally across all models of which models are censored on what topics
3 hours ago
It'd be interesting to use this technique to create a running tally across all models of which models are censored on what topics
Agreed, we find this to be an interesting reflection of societal values and norms inasmuch LLMs are.
A "free speech" benchmark already exists https://speechmap.ai/labs/
Yes but in which jurisdiction could you publish it? We roughly know what the hot topics are for the current models, but actually testing and ranking would break said censorship and thus would be hammered into the ground through cointelpro methods by all parties.
It would be nice to have a hypothetical small country where the internal censorship would be non aligned and insignificant enough that it wouldn't take away from the overall findings. But it doesn't exist.
I want some science based authority on the moon where only 3-sigma IQ international academics have ultimate authority. Oh wait Asimov did that right? I guess it didn't go so well either.
More important there are some things censored that are true. And some things censored that are false. How do we even get to a good model of the truthiness/nonsense adjustment indicator?
You could publish this in the US easily. 145 IQ peeps get tons of stuff wrong, btw, and in many domains my experience is the ‘wrongness’ can intensify as you move up into higher sigma domains.