Proxy IPs are also known and typically blocked. In fact, you can't even browse reddit without logging in when connected to most proxies.
Many web scraping companies have loads of phones hooked up in a rack in order to use mobile IPs. Companies can't just block mobile IPs because their site would become unusable for several city blocks (mobile IPs often correspond to a specific cell tower). This is the face of modern web scraping: https://i.imgur.com/U2RXi5G.jpeg
Just like every security feature in the physical and digital worlds, security just inconveniences honest people and the cost to bypass reduces the amount of people who try.
Eventually it becomes expensive to scrape reddit's data and most people will stop.
Could a motivated scraper not buy IPs/proxies that aren’t in those ranges, i.e. to blend in with general users?
Proxy IPs are also known and typically blocked. In fact, you can't even browse reddit without logging in when connected to most proxies.
Many web scraping companies have loads of phones hooked up in a rack in order to use mobile IPs. Companies can't just block mobile IPs because their site would become unusable for several city blocks (mobile IPs often correspond to a specific cell tower). This is the face of modern web scraping: https://i.imgur.com/U2RXi5G.jpeg
Or, if you're on the shadier parts of the web, hacked IoT devices put to good use instead of DDoS zombies.
Just like every security feature in the physical and digital worlds, security just inconveniences honest people and the cost to bypass reduces the amount of people who try.
Eventually it becomes expensive to scrape reddit's data and most people will stop.