Comment by leeches

Let's imagine a scenario:

There is a state of emergency presidential address. In Video A, the politician says X Y Z. In Video B, the politician says A B C. Both videos have equal credibility. The videos show no artifacts from tampering. The alteration is undetectable by experts. The broadcast has dire consequences in a divided country.

50% of channels are pushing Video A, 50% of channels are pushing Video B.

We are now in a position where the public actually cannot determine which video is authentic. The politician could broadcast a new statement, to clarify the validity of the first video. But, you could just as easily fake that too, to publish a statement that declares the opposite.

So, then you load up Hacker News or wherever, to determine for yourself what the hell is going on. But someone spins up 1,000 bots to flood the comments in favor of Video A, and someone else spins up 1,000 bots to flood the comments in favor of Video B. These comments are all in natural language, all with their own individual idiosyncrasies. It's impossible to determine if the comment is a bot. And because the cost is essentially free, these bots can be primed for years, making mundane comments on trivial topics to build social credibility. Actual humans only account for maybe 1% of this discourse.

Now imagine: our entire world operates like this, on a daily basis, ranging from the mundane to the dramatic. How about broadcasting a deepfake press statement from a CEO to make a shorted meme stock crash. If there are financial/political incentives to do so, and the technological hurdle is insignificant, these tools will be weaponized.

So how do we "not believe the media", do we all have to be standing in the same room together where something notable happens?

I understand that there could be upsides, the world isn't all doom and gloom. But, I think engineers get myopic, and do not heed the warnings.