Comment by MatthiasPortzel

3 years ago

No. Cloudflare offers different levels of protection. One level is ‘prevent DDoS.’ Another level is ‘prevent bots from accessing the site at all.’ Not all bots are part of a DDoS. The problem is that many website owners turn on the second setting, because ‘bots are bad,’ without realizing that this means that some of their users are going to have to fill out Captchas.

(Comment written from memory, I may have details wrong.)

8 comments

MatthiasPortzel

Reply
marginalia_nu  3 years ago

Sometimes it's a lesser evil. Clouflare blocks about 1.6 million bot search queries per day on my search engine. Simply could not operate it without this inconvenience.

  • justinclift  3 years ago

    > I'm currently looking for hosting for a large term frequency data file that is necessary for several of the search engine's core functions.

    Did you get that sorted out?

    Asking because we (sqlitebrowser.org, dbhub.io) have a bunch of Hetzner dedicated servers that are nowhere near fully utilised. Could probably figure something decent out using those, as Hetzner doesn't charge for bandwidth.

    • marginalia_nu  3 years ago

      Yeah that's solved itself, I eventually got a cheap VPS @ downloads.marginalia.nu for providing these files. It solves the immediate problem of hosting the data that can't go in git.

      How much space have you got by the way?

      2 replies →