Comment by TimorousBestie

6 months ago

LLMs broadly speaking disregard robots.txt, so I don’t see why they would bother regarding this.

0 comments