← Back to context

Comment by Symbiote

16 hours ago

Meta is an exception. They intentionally ignore robots.txt rules.

(No second chances, I blocked their whole IP range.)

3 comments

Symbiote

Reply
inigyou  13 hours ago

Everyone ignores robots.txt because every robots.txt blocks everyone.

  • nubinetwork  12 hours ago

    I allow Google, Bing, Yahoo, Kagi, and Apple on my personal blog... who else do I need to allow? Claude, OpenAI, Alibaba, and perplexity can go to hell.