Comment by dorkington

2 really seems to be the biggest issue, and then 7, 8

Most issues with grabbing web data isn't from grabbing it, it's from companies (who often scrape and harvest data at ENORMOUS RATES) turning around and making it difficult for people to do the same to them.

And then just the recursive nature of failed search. It seems to have a similar solution to dict brute force password guessing.