Comment by saghm
5 days ago
I think you buried the lede in your footnote here. Even if it is just a mistake, it's a pretty avoidable one by having a human in the loop to review changes to start blocking URLs to such a commonly linked site. If he thinks that it's "efficient" not to retain enough people to be able to notice that URL fragments and hashtags use the same symbol, he shouldn't be allowed anywhere near an "office of government efficiency", much less in charge of it.
Humans aren’t in the loop for automated bans. That has no relationship to staffing size.
This is likely a problem with the link banning algo not treating signal.me as high volume enough to prevent an automated ban.
That same logic most definitely exists at well-staffed companies and the internet is full of stories of people getting screwed by these systems. Google sinking legit companies with no recourse, locking out Gmail users who had decades of their life there, etc.
> Humans aren’t in the loop for automated bans.
My point is that this _shouldn't_ have been purely automated, but having fewer people to review things forces more things to be automated.
Well, I worked on abuse for a few years and gave a tech talk on it at Twitter. You really do want these things to be fully automated for two reasons.
The first is that spammers automate so if you don't then they're always much faster. By the time your humans are paying attention and have made a decision it's too late, all the spam (scams, frauds, malware sites...) was already delivered. In the next spam run everything will have changed, so, decisions made by that point are useless.
The second is that your suggestion contains an unstated premise that the human evaluators would have somehow more information to work with than an automated system, or would reach a different decision. In reality they don't and wouldn't. URL reputation systems like this are triggered by spam attacks. For a certain window of time there's a high probability that any message containing a specific domain name will be flagged by users as spam, so the system short-circuits that and starts classifying all messages of unknown status containing a link to that domain as spam. This works well because spammers usually want their targets to visit a website.
So the human evaluators in this case will see a message like:
"URL domain signal.me has 67% chance of spam and rising, confirm block? Y/N"
and the humans will always press Y because obviously (a) it means that such a decision is right more often than it's wrong and (b) the domain name is normally meaningless anyway. The block will be removed a bit later once the spammers go away.
In this case, it's tempting to think that some human in a cheap labor country would somehow see this message and think "ah! signal.me! clearly a domain linked to the super cool Signal messenger, which I personally like, so I won't block it even though this might cause a lot of people to be victimized by criminals". But they wouldn't and shouldn't. The domain even looks phishy, it's quite surprising to learn that it's a real Signal linked domain.
9 replies →
> This is likely a problem with the link banning algo not treating signal.me as high volume enough to prevent an automated ban.
And rightfully so. Despite being discussed a lot here, Signal is not very popular or well-known. Even TFA felt the need to start by explaining what Signal is. TFA then adds:
“This request looks like it might be automated” reads another prompt. “To protect our users from spam and other malicious activity, we can’t complete this action right now. Please try again later.”
And if signal.me is being used for automated spam, automated attacks require automated solutions.
> The real problem then is that even if it was deliberate (conspiracy theory: Mark messaged Elon: Pls help me curb the growth of the biggest competitor of Whatsapp?) twitter can easily hide behind "overzealous automation, sorry".
That would indeed be concerning. (And maybe illegal?) But is it anything more than an unfounded accusation?
[flagged]
[flagged]