I don't know if this falls under good ethical behavior but the one time I needed to mass scrape sites was to build an affordable housing directory for the Bay Area. Whether or not a unit of housing was under rent control was only available through private entities or apartment hosting sites and it was scattered and hard to find.
In my opinion, a directory of subsidized housing should have been provided by local governments and not through a plethora of real estate websites.
I don't know if this falls under good ethical behavior but the one time I needed to mass scrape sites was to build an affordable housing directory for the Bay Area. Whether or not a unit of housing was under rent control was only available through private entities or apartment hosting sites and it was scattered and hard to find.
In my opinion, a directory of subsidized housing should have been provided by local governments and not through a plethora of real estate websites.