← Back to context

Comment by Gigachad

2 days ago

There was a similar case where a web scraper was bypassing prevention mechanisms on linked in

https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn

That case is why Twitter, and anyone else with lawyers paying attention went and put content behind a login wall.

  • Twitter griefs everyone with a login wall because they want bulk downloaders to pay for API access instead and the login wall is an attempt to rate limit non-API bulk requests.

    That isn't relevant to ordinary media outlets because a) they don't have enough content volume for rate limiting to be effective since it's possible to get everything they publish even at a slow rate limit, and b) getting AI scrapers to subscribe to their bulk download API instead is not the objective in their case.