Comment by christkv

3 days ago

I fail to understand how is this different to a google web crawler in practice other than ignoring the robots.txt?

0 comments