Comment by rpastuszak

1 day ago

FYI I just tried to read your blog but I’m getting the: “Sorry, you have been blocked” banner from CF.

(Safari w. an adblocker, VPN off )

I find it incredible that people are willing to outsource the decision on who can access their sites to a corporation. Especially one that is increasingly dominant and tends toward worrying control (they also started selling paid search services for bots to use, presumably those provider will get access).

  • It's not unreasonable to leave a non-negligible effort which requires a bit of expertise to a specialist service.

  • It's a trade off. I use CFs load balancers to simplify steering to instances, but also appreciate that you get traffic filtering (even if not perfect). I think it ultimately depends on context and goals of what you're building. I wouldn't use that setup for everything, but I've found it's a healthy default for getting stuff stood up. Long-term, I think it makes a ton of sense to build up your own solutions.

  • If it's a big org doing the outsourcing, maybe there are people in the org who would rather not let others within the org have an accurate idea when the blocking efforts reach the point where the vast majority of desirable visitors are turned away.

    Everyone else just goes along for the ride if they want to enjoy the same convenience, as long as they can stomach it.

  • I find it incredible that people are willing to outsource the decision on what the kernel does on their server to a corporation.

I checked just for fun, also blocked. Could be because I'm in India, but even with CF warp, still blocked..

I do enjoy some of the free cloudflare services for personal projects, but to act like they provide some great infallible business structure or services is ridiculous.

  • Are any sites you'd expect to be on paid-for Cloudflare also blocked?

    I put https://www.gbif.org/ behind Cloudflare on the business plan, and so far (it's been roughly a year) we've only had a couple of reports of blocking, and both seemed likely to be caused by other users on the same IP scraping the site.

    • I can access gbif.org with and without WARP from here.

      Sometimes sites like hubspot block me without WARP, but it's usually small blogs or technical project sites that give me trouble.

      So it could have to do with free vs paid plans being more aggressive with default rules.

  • I’m blocked too. Wasn’t sure what you are talking about, but found the blog in the profile. I’m from my iPhone. I wonder which country they do not block.

Just tested it myself and got "This website is using a security service to protect itself from online attacks." from Firefox on Mac (with or without ad-blocker). What kind of online attack is expected on a blog?

  • Unfortunately almost any kind of blog. My spouse's blog, which she hasn't updated in several years, came under attack about a month ago. Her blog is just posts of some of our travels mixed in with her arts and crafts. It went from a few hundred hits a day to 20M hits a day coming from 2M unique "users" a day. It wasn't a full scrape, but it was hitting a broad range of pages. The traffic originated from 170 countries, so it would seem to be a botnet. It went on for 2-3 weeks before dropping below 1M hits per day. It still hasn't dropped back down to normal levels.

Same here, Firefox with ublock Origin and NoScript.

On the topic of caching I run my own edge nodes, no regrets. Privacy maintained, finer grained controls, no large scale ddos protection but I have never needed it.

  • Nice, what’s your setup? How much work was involved?

    • At the moment two large VM's using NGinx on the front-end and HAProxy on the back-end to do the load balancing and health checks. NGinx and HAProxy communicate with each other over a socket thus taking the network out of the picture. NGinx caches files to a tmpfs ram disk. The back-end HAProxy communicates over VPNs to the origin servers. I've automated scaling up edges but I still have work to do calculating the dynamic/static back-end limits per edge but for my tiny hobby sites I do not foresee needing to automate this any time soon. The edges force level 1 zstd which nullifies many of the bots that slip past my anti-bot methods as they just get gibberish. This also breaks old browsers which I do on purpose as nobody should be visiting a chan site (which I use as a replacement for my old blog) without a fully patched system. I keep additional VM's configured as spares or as "side doors" for friends to use in /etc/hosts in the event some day I attract h8 and vitriol. I use NGinx Geo in combination with FireHOL and a few other git repos to determine who should be allowed to post real time vs. be held in a moderator review queue which has saved me multiple times.

      I'm not doing Anycast or GSLB as it is just for a hobby site. If it ever grew in popularity (unlikely) then I would probably just do DNS GSLB in PowerDNS with PowerGSLB.

      For me it was not a lot of work because it is a hobby. I'm sure others here could automate this in Anisble, Docker, Podman and probably have. My setup is probably childs-play compared to what others here do professionally.

Are most people’s blogs getting so much traffic that they need this? I have a relatively heavy blog (it runs on Mediawiki) and I allow LLM agents to access it and everything. So far it seems all right. Perhaps I’m not writing anything interesting but the bots do tend of requests a second. Maybe others seeing more traffic than that?

Tbh. I think it's on purpose though. Blocked too and almost never encounter a block ( Chrome - Windows)