Comment by vwkd

With an AWS IP and a bot usage pattern they’ll surely ban your account pretty quickly or put you in front of a CAPTCHA. I wish it was as easy as a small script. Without anti-bot techniques, sites would be overflown by scraping bots. Try to scrape a Cloudflare protected site, for example. They’re really good in figuring out if you’re human or a bot. IIRC they even fingerprint your TLS handshake or cypher suite, which ultimately made me give up with headless Chrome and Puppeteer even after proxying through my residential IP, spoofing user-agent and screen size and rate limiting. Unfortunately, there’s no way to distinguish good bots for personal usage from bad bots.