Comment by mysteria

11 days ago

Even before AI was a thing some websites would deny all crawlers in robots.txt except for the Googlebot for the same reason.

0 comments