Comment by Kodiack
4 hours ago
As someone who hosts a few “tiny websites” that are “hiding behind Cloudflare”, it’s because of some incredibly misbehaved botnet traffic that’s otherwise persistently scraping via residential proxies.
This is a hobby for me. It’s for my enjoyment and it allows me to provide resources that others enjoy using. However, I’m not going to allow literally 99%+ of requests to be aggressive scrapers that won’t give up until you’ve got a heavy-handed solution in place.
I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.
> I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.
No, the actual alternative is learning to do it yourself. Unless you're a frequent target of UDP DDoS, there is almost nothing you can't defend against straight from the server itself.
Spending time learning how to configure your firewall, set rate limits in your web server, tune your application for caching and further rate limits, and should all this fail install a self-hosted captcha/proof-of-work fence yourself, is definitely a fair amount of efforts when those things aren't part of the core hobby.
Sorry, this just isn’t true. Some clients just don’t have the resources to pay for a dev playing whack-a-mole against residential proxy bots that are basically indistinguishable from legit users, but en masse can take down a VPS.
Rate limiting doesn’t work when individual IPs are only making a handful of requests.
So that’s why we’re using CF/Fastly. Too bad, I’d prefer not to, but there is just not an alternative at this point.