Comment by zigman1
2 years ago
To be honest, I see some positivity in that.
> Now it can be done cheaply, by anyone. It will become good enough that we cannot believe any form of media. And also make it impossible to determine if the written word is coming from an actual person. This is going to be weaponized against us.
We shouldn't believe any form of media straight away. We only do so because we think faking it is hard and why should one do. Being able to produce it cheaply could make people more attentative and skeptical of things around them. Blogspam sucks mostly out of consumers belief that this is something that was written by a person who deeply cares about them. Average internet consumer consumes shitty internet not because he is ignorant, but because he or she doesn't know enough to care.
But maybe I'm to optimistic, I just think people are not aware of stuff around them
Let's imagine a scenario:
There is a state of emergency presidential address. In Video A, the politician says X Y Z. In Video B, the politician says A B C. Both videos have equal credibility. The videos show no artifacts from tampering. The alteration is undetectable by experts. The broadcast has dire consequences in a divided country.
50% of channels are pushing Video A, 50% of channels are pushing Video B.
We are now in a position where the public actually cannot determine which video is authentic. The politician could broadcast a new statement, to clarify the validity of the first video. But, you could just as easily fake that too, to publish a statement that declares the opposite.
So, then you load up Hacker News or wherever, to determine for yourself what the hell is going on. But someone spins up 1,000 bots to flood the comments in favor of Video A, and someone else spins up 1,000 bots to flood the comments in favor of Video B. These comments are all in natural language, all with their own individual idiosyncrasies. It's impossible to determine if the comment is a bot. And because the cost is essentially free, these bots can be primed for years, making mundane comments on trivial topics to build social credibility. Actual humans only account for maybe 1% of this discourse.
Now imagine: our entire world operates like this, on a daily basis, ranging from the mundane to the dramatic. How about broadcasting a deepfake press statement from a CEO to make a shorted meme stock crash. If there are financial/political incentives to do so, and the technological hurdle is insignificant, these tools will be weaponized.
So how do we "not believe the media", do we all have to be standing in the same room together where something notable happens?
I understand that there could be upsides, the world isn't all doom and gloom. But, I think engineers get myopic, and do not heed the warnings.