Comment by beejiu

7 years ago

When I recently added 'click to unsubscribe' functionality to my emails, the URL got wrote out into some logs. Those logs got written to a Slack channel and Slack loves to click any link it sees. Oh, and it doesn't respect robots.txt.

But all I saw was every member of my list clicking 'unsubscribe'. It took a good hour to figure out exactly what was going on.

Idempotence is not the problem here, by the way. That just means calling the method twice has the same effect. But GET should have no side-effect, in an ideal world. Of course, in the case of unsubscribe links, it needs to have a side-effect to comply with the law.

83 comments

beejiu

Reply
tialaramex  7 years ago

Actual public Certificate Authorities have done this too.

You request a cert, it's authorized everything seems fine. Except, huh, the guy who was supposed to authorize is off sick today, how did that work? The email to the authorizer should just be sat in his INBOX until he gets back.

Oh - the company's "Malware protection" system automatically dereferenced the "Do you want to issue this certificate?" link from the email and there was no second step. So for affected companies basically anybody could request any certificate in their domains and it would get issued.

As far as I remember nobody has proof any bad guys ever used this, but grey hats posted some fun they had with it. Likewise for a CA that decided to OCR the images from an unco-operative DNS hierarchy that wouldn't provide machine readable data to them. Grey hats obtained domain names that confused the OCR into allowing them to get certs for other people's names. Did any black hats do it? We have no proof.

  • kingbirdy  7 years ago

    Any chance you could provide articles on these grey hat activities? Sounds interesting

    • tialaramex  7 years ago

      For the OCR one the combination of "WHOIS" and "OCR" found me this thread from 2016 with an incident report from Comodo's Robin Alden:

      https://www.mail-archive.com/dev-security-policy@lists.mozil...

      Alas I wasn't able to bring to mind a combination of keywords that would find the other incident in public archives, and I know it's from my background reading so it will be before my personal archives of these discussions begin. Sorry.

bjackman  7 years ago

Aren't you allowed to have your "click to unsubscribe" button lead to a page with a button that does a POST that actually unsubscribes? I feel like I've seen that approach in use.

  • StavrosK  7 years ago

    What they do is load a page with a form redirect to do the POST, I believe, so link loaders won't follow it and you'll be safe.

  • drdeca  7 years ago

    How about just having, near the unsubscribe link in the email, a link that says "click here to ignore up to one unsubscribe link press within two minutes of clicking this link", so that the automated process that clicked the unsubscribe link by mistake will also click that link.

    The page in question could also have a "actually, I do want to unsubscribe, I just clicked the wrong link by accident" button as well, in case a human reader is confused.

    ---

    Or, one link that says "unsubscribe immediately" and another that says "unsubscribe only after confirmation", and the first one unsubscribes immediately, unless they also click the second link immediately before or after, while the second link only unsubscribes them if they click a confirm button on the page?

    ---

    Or, maybe the unsubscribe link could have a confirmation button, but would also have some javascript to confirm the unsubscribe after a few moments of the page being fully loaded (the button being used if they have javascript disabled, for example)

  • criddell  7 years ago

    Don't make your users jump through hoops to unsubscribe. That seems like a typical dark pattern to me.

notheguyouthink  7 years ago

> Idempotence is not the problem here, by the way. That just means calling the method twice has the same effect. But GET should have no side-effect, in an ideal world. Of course, in the case of unsubscribe links, it needs to have a side-effect to comply with the law.

Thank you! I felt like I was taking crazy pills with my understanding of idempotence.

Calling GET /door/open is idempotent too, but it's still gross in my opinion. The author makes it sound like that would be fine.

  • chaosmonkey  7 years ago

    I think he only had GET /door/toggle, which cannot be idempotent.

    • notheguyouthink  7 years ago

      Yea, but his point was that GET requests should be idempotent.. when that's not the important part of GET. That's why I used the example of GET /door/open, because that is idempotent, but it's still gross - it's causing a state change from a GET request while still being idempotent.

      GET being idempotent is not the issue the author was dealing with. State change from a GET is the real issue.

    • luma  7 years ago

      He mentions later in the thread that he doesn't have a sensor to indicate door position, which is going to kill this project and absolutely prevents an idempotent approach. There are a bunch of ways to approach this, most of which are patented by Chamberlain, which is the specific reason you don't find many garage automation solutions sold as a bundle in the US.

      Typical bolt-on solutions would be a range finder pointing down from the ceiling near the door which can tell if the door is obstructing it, a tilt sensor mounted to the top door panel, or limit switches on the carriage way/tracks. With a door position sensor in place a simple momentary-contact dry relay can be used to trigger door motion.

      Once that is in place, you can add another WeMos D1 mini to your car to open and close the door with no interaction: https://github.com/aderusha/MQTTCarPresence

      4 replies →

sean-duffy  7 years ago

Facebook Messenger has the same problem, for a while my friends and I couldn't figure out why our referral links for a service weren't working, turns out Messenger was "using them up" for us.

cryptonector  7 years ago

Consider link shortener services...

It's perfectly fine to have certain side-effects, provided the GET is idempotent, but not others/most. Specifically, it's fine to idempotently have side-effects where it doesn't matter who is causing the effect, the side-effects are desirable, and the side-effect load won't be overwhelming.

In the case of a link shortener one can pretend that the side effect did not happen the first time the service sees some particular link, that the shortening has already occurred (at the beginning of time!). There is definitely a side-effect, since it involves updating a persistent hash table, and it is idempotent (though one could construct a shortener where it's possible to get more than one shortened form for a given URI when racing to shorten it, but this is not a problem for this particular sort of service).

thomasz  7 years ago

“Toggle” is by definition not idempodent, because you get a different result each and every time. “Open” and “close” are idempodent, but not safe. The result of a GET request should always be idempodent and safe.

  • dewiz  7 years ago

    GET requests should have no side effects. In other words NOOP is idempotent

    • jimktrains2  7 years ago

      Get should be safe, but that's not the same as indempotent. Delete requests are also indempotent as the final state will always be the same.

    • bad_user  7 years ago

      Reading from a database isn’t “side effect free”, neither is reading from a mutable variable.

      Given that HTTP requests are supposed to trigger database reads that may return a different result after some time, then “side effect free” would indeed be incorrect.

      The litmus test is ... does it always return the same output given the same input? If that can change, the it’s not describing a pure function, but a side effectful one.

      And the output of a GET request can and does change.

      Speaking of which people here aren’t talking of idempotency in a mathematical sense:

      f(f(x)) == f(x)

      If you really think about it, GET requests aren’t even idempotent ;-)

      2 replies →

eddz  7 years ago

> But GET should have no side-effect

I disagree. There are some ways in which GET can be non-idempotent, such as pageview counters and endpoints with a vast amount of constantly-changing content, for instance. One may argue that the first example may be possible with a GET followed by a POST, but any subsequent GET response (assuming it contains the counter) would still be different to its prior.

javajosh  7 years ago

It would be sufficient to GET a resource that uses a script to POST the side-effect. Slack's user-agent probably isn't sophisticated enough to mess this up (although heaven help us when they implement their preview with something like headless chrome).

  • poizan42  7 years ago

    What would the problem be in just showing a button that says "Confirm unsubscribe" that sends a POST request? A lot of sites does something like that for their newsletter unsubscription.

athenot  7 years ago

I remember having a similar issue in 2000, before any (meaningful) client-side javascript. Solution: each link that did something also had a request_id parameter, which was a timestamp in milliseconds. Two requests with the same request_id meant the user had clicked something twice, so any action would NOT be performed multiple times if the same request_id came in more than once.

This let users double click on links and have the action performed only once. In 2000, hyperlinks were still confusing to some users who were used to "double-click = open", especially for file icons.

EDIT: added text in italics because initial wording was confusing.

  • ams6110  7 years ago

    A not-small percentage of users double click links (and buttons, and anything else that needs clicking).

    • 0xcde4c3db  7 years ago

      The wording is a little ambiguous, but I think the scheme was that the first request is honored and any subsequent requests are ignored, not that additional requests cancel or undo the action altogether.

      1 reply →

thomasahle  7 years ago

> Idempotence is not the problem here, by the way. That just means calling the method twice has the same effect. But GET should have no side-effect, in an ideal world. Of course, in the case of unsubscribe links, it needs to have a side-effect to comply with the law.

Could you have the page redirect to itself with POST? Like javascript redirect or metatag. Browsers would do this, and there wouldn't be any different for users, but bots, slack and Safari probably wouldn't.

RandomCSGeek  7 years ago

The problem is not that your endpoint had side effect. Rather, it didn't have any side effect(it is supposed to unsub, and it does exactly that).

The problem is with lack of authentication. Slack's ability to unsubscribe people on their behalf, without their explicit permission seems to be the real issue here.

Even a "I'm not a bot" check would provide some protection.

amatheus  7 years ago

If the unsubscribe link were unique for each subscriber would the law still be satisfied?

  • wlll  7 years ago

    I thought that initially, but then I considered that for users to be bulk unsubscribed then the link would surely have to be the same for every user, at which point the each user gets the same unsubscribe link, and then when they click on it it unsubscribes everyone.

    This doesn't seem very likely, so I guess that a whole load of unique unsubscribe links got dumped into slack which started following them.