Comment by dwaite

> As the botmakers circumvent, new methods of proof-of-notbot will be made available.

Yes, but the fundamental problem is that the AI crawler does the same amount of work as a legitimate user, not more.

So if you design the work such that it takes five seconds on a five year old smartphone, it could inconvenience a large portion of your user base. But once that scheme is understood by the crawler, it will delay the start of their aggressive crawling by... well-under five seconds.

An open source javascript challenge as a crawler blocker may work until it gets large enough for crawlers to care, but then they just have an engineer subscribe to changes on GitHub and have new challenge algorithms implemented before the majority of the deployment base migrates.