← Back to context

Comment by arielcostas

1 year ago

A lot of people are failing to conceive the danger that poses to the open web the fact that a lot of traffic runs through/to a few bunch of providers (namely, CloudFlare, AWS, Azure, Google Cloud, and "smaller" ones like Fastly or Akamai) who can take this kind of measures without (many) website owners knowing or giving a crap about.

Google itself tried to push crap like Web Environment Integrity (WEI) so websites could verify "authentic" browsers. We got them to stop it (for now) but there was already code in the Chromium sources. What makes CloudFlare MITMing and blocking/punishing genuine users from visiting websites?

Why are we trusting CloudFlare to be a "good citizen" and not block unfairly/annoy certain people for whatever reason? Or even worse, serve modified content instead of what the actual origin is serving? I mean in the cases where CloudFlare re-encrypts the data, instead of only being a DNS provider. How can we trust that not third party has infiltrated their systems and compromised them? Except "just trust me bro", of course

> Or even worse, serve modified content instead of what the actual origin is serving?

I witnessed this! Last time I checked, in the default config, the connection between cloudflare and the origin server does not do strict TLS cert validation. Which for an active-MITM attacker is as good as no TLS cert validation at all.

A few years ago an Indian ISP decided that https://overthewire.org should be banned for hosting "hacking" content (iirc). For many Indian users, the page showed a "content blocked" page. But the error page had a padlock icon in the URL bar and a valid TLS cert - said ISP was injecting it between Cloudflare and the origin server using a self-signed cert, and Cloudflare was re-encrypting it with a legit cert. In this case it was very conspicuous, but if the tampering was less obvious there'd be no way for an end-user to detect the MITM.

I don't have any evidence on-hand, but iirc there were people reporting this issue on Twitter - somewhere between 2019 and 2021, maybe.

  • Cloudflare recently started detecting whether strict TLS cert validation works with the origin server, and if it does, it enables strict validation automatically.

I can easily conceive the danger. But I can directly observe the danger that's causing traffic to be so centralized - if you don't have one of those providers on your side, any adversary with a couple hundred dollars to burn can take down your website on demand. That seems like a bigger practical problem for the open web, and I don't know what the alternative solution would be. How can I know, without incurring any nontrivial computation cost, that a weird-looking request coming from a weird browser I don't recognize is not a botnet trying to DDOS me?

  • Exactly. If you're going to bemoan centralization, which is fine, you also need to address the reason why we're going in that direction. And that's probably going to involve rethinking the naive foundational aspects of the internet.

  • how do you know a normal-looking request coming from google chrome is not a botnet trying to ddos you?

    • You deploy complex proprietary heuristics to identify whether incoming requests look more like an attack or more like something a user would legitimately send. If you find a new heuristic and try to deploy it, you'll immediately notice if it throws a bunch of false positives for Chrome, but you might not notice so quickly for Pale Moon or other non-mainstream browsers.

      (And if I were doing this on my own, rather than trusting Cloudflare to do it, I would almost surely decide that I don't care enough about Pale Moon users to fix an otherwise good rule that's blocking them as a side effect.)

I don't think people aren't aware that it's bad. They just don't care enough. And they think "I could keep all this money safely in my mattress or I could put it into one of those three big banks!" ... Or something like that.

Maybe it's the customers I deal with, or my own ignorance, but what alternatives are there to a service like Cloudflare? It is very easy to setup, and my clients don't want to pay a lot of money for hosting. With Cloudflare, I can turn on DDoS and bot protection to prevent heavy resource usage, as well as turn on caching to keep resource usage down. I built a plugin for the CMS I use (Umbraco - runs on .NET) to clear the cache for specific pages, or all pages (such as when a change is made to a global element like the header). I am able to run a website on Azure with less than the minimum recommended memory and CPU for Umbraco, due to lots of performance analyzing and enhancements over the years, but also because I have Cloudflare in front of the website.

If there were an alternative that would provide the same benefits at roughly the same cost, I would definitely be willing to take a look, even if it meant I needed to spend some time learning a different way to configure the service from the way I configure Cloudflare.

  • What's the cost of annoying people trying to browse to your sites, some to the point where they'll just not bother?

    • This is rather blunt, but if it is between 98% (CF-protected) versus near-0% (heavily-DDoSed site), then you hopefully you now see the dilemma that other people faced.

      2 replies →

    • For companies that are just built around a marketing funnel to provide enough info to get you to fill out their contact form to sell you something, my guess is that Cloudflare is well worth the cost over increased hosting fees. I know it's not the answer anyone wants to hear, but I don't deal with too many companies selling anything more than around 5 or 6 figures, with products that you don't necessarily need very often.

      I would like to know if there are alternatives somewhere close to the same cost, where I don't need to use Cloudflare. I don't enjoy annoying customers, or even dealing with sales and marketing, but I have built lots of software where I get to control the technology, and can get a new website up and running in 3 hours, with a ton of built-in functionality. I've spent about 12 years reducing the amount of memory the Umbraco CMS uses, compared to normal installs, and I love that aspect of my career. If I could get my clients to pay more and not use Cloudflare, I would happily go that route, believe me!

Of course we're trusting CloudFlare to be a good citizen. If they were not, they would be banned - unless they sold their business to a sovereign wealth fund.

  • I don't get if this is sarcasm (perhaps a reference to TikTok?), but in my case (european) it's a foreign third-party for me