Comment by denschub

19 days ago

> Is it all crawlers that switch to a non-bot UA

I've observed only one of them do this with high confidence.

> how are they determining it's the same bot?

it's fairly easy to determine that it's the same bot, because as soon as I blocked the "official" one, a bunch of AWS IPs started crawling the same URL patterns - in this case, mediawiki's diff view (`/wiki/index.php?title=[page]&diff=[new-id]&oldid=[old-id]`), that absolutely no bot ever crawled before.

> What non-bot UA do they claim?

Latest Chrome on Windows.