Comment by sph

3 years ago

I'm working on a crawler and CloudFlare is the cause of 99% of all the headaches and random bugs I encounter doing simple HTTP requests.

I literally have implemented custom logic to deal with sites returning the "Server: Cloudflare" header.