Comment by weird-eye-issue
12 hours ago
You are conflating a couple of different things here
There actually is such a thing as verified bots on Cloudflare that gets through most blocks (and these services are likely are part of that), but ultimately it just depends on how the website owner has things set up in Cloudflare
> There actually is such a thing as verified bots on Cloudflare that gets through most blocks
Verified bot is just a label. What you do with that information is entirely up to you as the operator. It doesn't say anything anything about the service and doesn't provide any guarantees about the traffic.
No it's not just a label because it's a category that gets used in the firewall settings. A lot of sites allow only verified bots and block non-verified ones. So yes as I already mentioned website owners can of course block verified bots but it's much much less common than blocking non-verified ones because most site owners don't want to accidentally deindex their site from Google...
You get to decide per bot. Or per category: https://developers.cloudflare.com/bots/concepts/bot/verified... You can make for that Google or all indexers are always allowed.
1 reply →
I hate to be rude but once again I am begging people to actually read the post. It is mentioned in the second paragraph that they are all verified bots.
Actually it doesn't say they are verified bots but it does say that they follow the verified bots standards. But yes I think we can assume they are verified bots which I already said in my comment so I'm not really sure what the purpose of your reply is?