Comment by data-ottawa
9 hours ago
reCaptcha is a pretty strong wall to allow only Google to index websites, especially now that you need device verification. Throw in Cloudflare too.
There’s not much room to squeeze in when your competitors hold the keys to 15 million top websites.
I write a lot of scrapers. Both of those are pretty trivial to bypass at scale.
What about not at scale?
I find it wild that "at scale" we can bypass anti-bot measures, but just "normal" internet use (i.e Non-Google Browser or VPN) will throw a million captchas at you.
cgnat is pretty bad too.