Comment by hipadev23

3 months ago

Robots.txt is completely irrelevant. TOU/TOS are also irrelevant unless you restrict access to only those who have agreed to terms.

"By accessing this Site, you acknowledge that you have read, understand, and agree to abide by the terms described herein."

<https://www.imperva.com/legal/website-terms-of-use/>

Many, many, many hits for this or similar language:

<https://duckduckgo.com/?q=%22By+accessing+this+Site%2C+you+a...>

Mind: just because it's written doesn't mean it's enforceable, but to argue that what you've just denied isn't a widely-used premise of online contracts of adhesion fails the simplest empirical test.