Comment by mirzap
3 days ago
We like to think surveillance is something you can turn on for one problem and turn off afterward. In practice, that never happens. Once the machinery is in place, it stays and looks for new work. Tools justified today by "illegal immigration" won’t stop there. They drift into credit scoring, health insurance pricing, hiring and firing decisions, school admissions, housing access, travel permissions, banking, welfare eligibility, and even which online accounts are allowed to exist. Not because anyone set out to build a dystopia, but because systems, once built, naturally expand to whatever can be measured and enforced.
As Benjamin Franklin put it: those who give up essential liberty for temporary security deserve neither. The tradeoff rarely feels extreme at the time. It feels reasonable. By the time it isn’t, there’s no way back.
There is a gradual chilling effect of self-censorship to mass surveillance and loss of anonymity. When you know you are being watched, you change your behavior. You don't visit the "wrong" protest, you don't meet with the "controversial" whistleblower, and you don't seek out the "unpopular" doctor. Total surveillance creates a "soft" totalitarianism where citizens police their own movements to avoid falling into a "high-risk" algorithm, even if they've done nothing illegal. At its extreme, such societies end up with no outliers, no more of "the crazy ones. The misfits. The rebels." (Steve Jobs). Safety and compliance at all cost.
The peer-reviewed consensus of this in psychology describes a three-step internal process of Anticipatory Anxiety, Risk Aversion and Self-Censorship [1]. The Conforming Effect (Conformity Theory) has been measured in studies such as those by Jonathon Penney (2016/2021), where use of Wikipedia data and search traffic shows a statistical drop in "sensitive" searches (e.g., about "terrorism," "human rights," or "health") immediately following news of government surveillance. [2]
[1] Surveillance as a Socio-Technical System: Behavioral Impacts and Self-Regulation in Monitored Environments, https://www.mdpi.com/2079-8954/13/7/614
[2] Chilling Effects: Online Surveillance and Wikipedia Use, https://lawcat.berkeley.edu/record/1127413?v=pdf
Yup, I agree. And this is why I think mass surveillance isn’t just another technology to regulate. The chilling effect isn’t misuse; it’s the default: continuous, opaque observation changes behavior by itself. Because it’s centralized and unavoidable, people self-censor and conform; you don’t need arrests once everyone assumes they’re being scored.
We don’t yet have long-run examples of fully algorithmic surveillance societies, so the outcome isn’t certain. But if these dynamics scale, the risk is trading experimentation for legibility. Problems get hidden, metrics look clean, and warning signals vanish. When real stress hits, responses are late and blunt - overcorrection, cascading failures, accelerated exit. Stability holds until it doesn’t.
I think especially heinous is the use of Zero-Knowledge (ZK) Proof technologies where a centralized attestation authority (e.g. government agency) verifies compliance, and the verifier (e.g. business needing to prove compliance) relies on the ZK cryptographic proof of compliance without revealing the individual. This revocable privacy can unmask the real identity in the case of asserted "suspicious" activity. This is the current direction of mainstream technology, and all it serves to accomplish is a normalization of loss of privacy and anonymity.
2 replies →
>Total surveillance creates a "soft" totalitarianism
And every step of the way the enablers will defend it on the grounds of "well you still technically can do the thing if you're willing to put up with some absurd risks or jump through some insane and impractical hoops specifically designed to be non-starters for many/most."
This specific point is addressed in a famous 1995 anti-technology essay by Ted Kaczynski.
Specifically paragraphs:
127. A technological advance that appears not to threaten freedom often turns out to threaten it very seriously later on. ...
128. While technological progress AS A WHOLE continually narrows our sphere of freedom, each new technical advance CONSIDERED BY ITSELF appears to be desirable. ...
129. Another reason why technology is such a powerful social force is that, within the context of a given society, technological progress marches in only one direction; it can never be reversed. ...
https://www.washingtonpost.com/wp-srv/national/longterm/unab...
Who ever said facial recognition wasn't going to threaten freedom? None of those points feel at all relevant or substantive to the topic of discussion
Most of people have no idea and are totally ok with being tracked 24/7
[dead]
Melodramatic slop from the original edgy school shooter. There are plenty of technologies that increase freedom. For example, I am substantially more free to not die of smallpox, which would have been quite limiting to my options.
> Melodramatic slop from the original edgy school shooter.
This is a very arrogant, judgemental, dismissive comment that adds nothing to the conversation. It is also a textbook example of ad hominem. “Why are you paying attention to what that guy said?”
> There are plenty of technologies that increase freedom. […For example. Smallpox vaccine…]
If you think that Kacynski or OP were talking about all technologies then you lack reading comprehension. Since they’re not making the assertion about all technologies, holding up a specific technology as being good does not address the point that was being made.
> from the original edgy school shooter.
Regardless of your views on Kacynski, he is a philosopher of note. His work is regularly quoted and referred to 30 years later. As opposed to, say, Bin Laden’s manifesto.
> Melodramatic slop
It’s ironic that you chose this phrasing, when “slop” has come to mean “low effort, low quality content pushed out without much thought”.
How humiliating for you, to put your foot in the mouth in front of everyone in this distinguished forum. This isn’t Digg, or even Reddit. Put some thought into what you write.
18 replies →
I don't think this is a useful framework for understanding these issues. What you are saying can, in essense, boil down to "any law enforcement is bad". ICE and its inhumane practices are just symptoms of an increasingly authoritarian administration that receives sufficient mandate from the population to push for increasingly authoritarian practices. The tools are just that, tools. The situation will keep getting worse until the population gets sick of it enough to push the wannabe autocrats out of power (and not replace them by other wannabe autocrats), and have the new administration dismantle these tools. Easier said than done, I know.
I think we're finally seeing the culture that's been present in law enforcement forever playing out to its logical end. The solution is pretty close to "all law enforcement is bad". We're seeing that the people most prone to violence and abuse seek out positions of power in law enforcement. Basically anyone who wants to be a cop should not be allowed to be a cop.
We can blame autocrats while also blaming the complicit tools. In Grand Rapids, Michigan yesterday the local police arrested the organizer of a protest against the invasion of Venezuela while she was on camera interviewing with the local news for "obstructing a roadway" (marching in a lane with other lanes open to traffic) and "disobeying a lawful command".
When we have local beat cops colluding with national secret police and suppressing dissent, we have a very serious problem and are running out of options very quickly.
I usually fall in the "all law enforcement is bad" part of the population too. But I don't think law enforcement in any society has ever not attracted these type of people you describe. I believe the "culture that's been present in law enforcement forever" you describe is completely unchanged.
To me, the more interesting question would be: why is law enforcement getting away with so much as of late? And the answer ties back to the current administration and the signicant part of the population behind them. If so many americans weren't cheering on ICE and cie., none of this would fly and it would blow over almost immediately. You get authoritarianism when authoritarian thinking wins. Authoritarian thinking wins when complex socio-political and socio-economical reasons I don't care to go into today.
The main thing I'm trying to say here, I guess, is that I reject the slippery slope fallacy ("get age verification today, get 1984 tomorrow"). If you want to fight authoritarian practices, find their source and fight that instead (the "how" is left as an exercise to the reader).
That first sentence of yours really struck a chord with me. I tried to think of other examples:
Cars — essential for leveraging time to travel longer distances and carrying multiple passengers and heavy loads; ens up being used by one person to drive three minutes to get coffee.
Guns — to quickly précis a … complex topic: good guys, but also bad guys.
Electricity — power generation goes up decade after decade, but so too does consumption with wasteful consumption going hand in hand with productive consumption.
As you might be able to tell, I think the answer to the question “how do we stop technology X from destroying us?” lies in licensing and regulation enacted through legislation.
I think those examples miss an important distinction. Cars, guns, and electricity are consumer technologies. They’re widely distributed, regulated, and constrained by market forces and law. Individuals can choose how to use them, and misuse is at least partially visible and contestable.
Surveillance is different. It’s inherently centralized and asymmetrical. By design, it gives one side - the state or large institutions - persistent visibility into everyone else, with little reciprocity. You can regulate how it’s used on paper, but the power imbalance remains.
It’s closer to nuclear technology than to cars or electricity. I can’t build a nuclear weapon or possess fissile material, not because it’s inefficient, but because some technologies are considered too dangerous to be broadly accessible. Mass surveillance belongs in that category. Once it exists, citizens don’t get to opt out, and meaningful oversight tends to lag far behind capability.
Licensing works when the technology is decentralized. With surveillance, the risk isn’t misuse at the edges - it’s concentration at the center.
clothing i think is a big one. Once the poster-child of industrialisation, now results in millions of tons being thrown away each other at a massive environmental cost.
> I think the answer to the question “how do we stop technology X from destroying us?” lies in licensing and regulation enacted through legislation.
In the golden age of the 90's we were able to ban CFCs, but I'm skeptical we could do that today. We no longer have that political ability, and I doubt we will get it back any time soon.
Your comment should offend far more on HN than it will.
Heck, drop into any comment section about transportation infrastructure or environmental policy (or a few years ago public health policy as well) and there's all sorts of evil mustache twirling going on about how to use basically the same sort of technologies to deploy state violence in pursuit of some goal and they are either unable or unwilling to think a few steps ahead see that what they're advocating for will over time if not quickly lead to dark places as policy and priorities change incrementally.
As I'm concerned the people who are happy to peddle this stuff when it suits them are just as complicit as the people who are cheering for it right now when it's being used for "obviously bad" things.
>As Benjamin Franklin put it: those who give up essential liberty for temporary security deserve neither. The tradeoff rarely feels extreme at the time. It feels reasonable. By the time it isn’t, there’s no way back.
This quote is like a lightening rod for exactly the kind of people I'm talking about.
Indeed, that's true. Payment for autism, originally intended for sick children, now a Somali scam. Veteran's disability, originally a means to allow people who were injured while serving the country, now a way for a desk-jockey to receive an annual stipend.
Any mechanism, once built, seeks to expand its scope. Until it delivers mail ;)