Comment by ToucanLoucan
5 hours ago
> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through.
Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop.
Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.
The article also mentions that a ton of even worse ads were posted between when they contacted Meta and when they were ready to go live with the article, and some of the offending ads which were taken down had just been re-uploaded again and automatically approved.
Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.
I think people have been demanding this for ages but lobbying by social media companies against any change in the law is enormous.
As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.
More specifically, these were for ads. They received compensation to distribute this matierial.
FB might or might not have a right to exist in someone's eyes, but the question remains: is 0.5c-like latency messaging across the world acceptable? Because facebook or not, given internets this is what you will end up with. People are people.
On the other hand, now try to ban that telecom. Gods will laugh.
Yeah and/but also the case it's not bidirectional enough
The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down
> The eternal question of tech platforms - should we be responsible for what users post?
I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible.
Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts.
Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
Yeah sure, I think we basically agree
I'm just bemoaning the fact FB can't even pretend they care when they're so busy banning people for nothing
2 replies →
But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
---
This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
>But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
you say sarcasm, but i think this is literally true.
this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
Don’t they have like an 80% gross margin?
6 replies →
"While remaining a going concern."
Well, tough. Maybe they should have to find a way. If that means requiring everybody to have to pay to afford the moderators then so be it. All the other companies would have to as well.
1 reply →
The point I'm making is they're not entitled to scale a business if they can't scale responsibly
1 reply →
I agree and the obvious solution is: Nothing should exist at that scale. In fact, "Able to effectively moderate your product," strikes me as an excellent standard to hold them to, then let their product find the natural usercount and price point they can manage.
Who ever said that companies should have the right to exploit network effects to dominate online communications?
Facebook absolutely could remain a going concern while dealing with all of these problems, the problem is that they would be spending money to lose money.
One of the more perverse aspects of risk prevention is that riskier customers are often way more lucrative. It's kind of inherent to the notion that markets price risk into transactions. The thing is, illegal behavior is also just a particular kind of risk: if someone wants to sell nudify apps or scam old people, Facebook could conceivably be on the hook[0] for damages if they're caught. And in a sense, a risk premium is just a fancy kind of bribe: if Facebook won't take my ads because they think they're too risky, then I'll just pay more and more until Facebook accepts them.
With illegal behavior, we hope that we can make the punishment high enough that Facebook refuses any conceivable risk premium the perpetrators of that behavior would be willing to pay. The problem is that Facebook is really desperate for any increasing growth story, and when you're too big to fail, your growth story is to drip-feed enshittify your stack. Increasing tolerance for illegal behavior is downstream of this: you cut enforcement, tell them to automate more, they become easier to bypass... and you make more money. Because it turns out all that risky behavior... is only risky for normal people. Facebook can launder[1] the risk away and make a tidy profit.
[0] Do not tell me about CDA 230, I was there when the old magic was written.
[1] See also, HSBC turning themselves into a literal money laundering operation.
1 reply →
We used to say computers can't make decisions because they can't meaningfully be held responsible for those decisions. Now the vibe seems to be that computers should make decisions because instead of anyone being held responsible we can just say "the computer made a mistake and it is very sorry" and effectively shield anyone from being responsible. But the thing is, machines don't do stuff. Not from a responsibility standpoint. My car doesn't drive, I do. It may provide the means, but I decide where we're going and how we're going there. Even with the active cruise on, the car didn't speed, I did. The car may have hit the wall, but I caused the accident. the car ran that person over, but I killed someone, not the car. The machine did what someone told it to do. If someone told it to do a bad thing, or someone did a bad job telling it not to a bad thing, that person is responsible.
When people started using Grok to make CSAM the platform's response was to paywall image generation. If we don't start making the driver responsible for what the car does, the coder responsible for what the code does, the CEO responsible for what the company does, they will continue to exploit people in increasingly horiffic ways.