Comment by baby_souffle

I don't know who 'they' is here but that's not the point? I would bet that a decent chunk of scraping happens because there is no API (or other machine-focused interface, like RSS).

"pay to crawl" sounds like the absolutely laziest possible way that a particular site could bolt on an API.