Comment by alt187

5 months ago

The tarpit is made for LLM crawlers who don't respect robots.txt. Do you love LLMs so much that you wish that they wouldn't have to respect this stupid, anticorporate AI-doomer robots.txt convention so they can pry out of the greedy hands of the webserver one more URL?

Maybe you just had a knee-jerk reaction.