Comment by Velocifyer

3 months ago

Try using anubis. It uses a PoW challenge to make it not make economic sense to scrape websites.

Anubis is trivially bypassed by anyone that cares to bypass it. All it does is inconvenience real users with niche/older/extended browsers or those who take basic precautions against tracking and malware.

Anubis won't work now that scrapers just allocate more CPU time to beat Anubis challenges. The default configuration also permits all bots, only catching bots pretending to be browsers.