Comment by contravariant

If it's unethical it's not because of what the robots.txt says.

Blindly violating it is bad manners, but deliberately scraping a single website over a month isn't the worst.