Comment by humanji
9 days ago
I’m coming at this from a founder/product angle, not a technical one, so excuse the naive framing.
What worries me isn’t scraping itself, but the second-order effects. If large parts of the web become intentionally unarchivable, we’re slowly losing a shared memory layer. Short-term protection makes sense, but long-term it feels like knowledge erosion.
Genuinely curious how people here think about preserving public knowledge without turning everything into open season for mass scraping.
This partially feels like an intentional pendulum swing from Twitter/Facebook cancel culture and other forms of policing.
I'm thinking in particular about the rise of platforms like Discord where being opaque to search/archiving is seen as a feature. Being gatekept and ephemeral makes people more comfortable sharing things that might get a takedown notice on other platforms, and it's hard for people who don't like you in the future to try to find jokes/quotes they don't like to damage your future reputation.
Clearly very different than news articles going offline, but I do think there's been a vibe shift around the internet. People feel overly surveilled in daily life, and take respite in places that make surveillance harder.