Comment by DevKoala

But think about it. Will you do it for your web property? Is someone else going to do it for my web property when I have clearly blocked robots? Will I do it for another web property for my agent to work and hope they don’t update their design or protect themselves against it?