← Back to context

Comment by delusional

20 days ago

The opposite of this.

Do be angry at the people misusing the systems. Don't be angry at the people building them for good.

27 comments

delusional

Reply
goda90  20 days ago

If someone points out that the system you're building can be abused, and you don't stop and come up with a solid plan to prevent abuse then you're just building the system for abuse.

  • delusional  20 days ago

    It's practically impossible to build a system that "can't be abused". If you set the bar there, then you can block any policy forever by simply enumerating increasingly unlikely ways for it to be abused. It's like a child's version of politics.

    I could go into my car right now and plow through a bunch of people. I'm still allowed to own a car. We've made the actual harmful act illegal, not the thing that theoretically made it possible.

    • davrosthedalek  20 days ago

      At the same time, we do not allow people to have nuclear bombs.

      As everything in life, it's a trade-off, but a good trade-off can only be found if people are fully aware of the consequences. It seems to me, people regularly underestimate the negative consequences of data collection (or realize that these consequences will not affect them, but others).

    • JumpCrisscross  20 days ago

      > It's practically impossible to build a system that "can't be abused"

      For ALPRs? I’d make queries public with a short delay, including with a unique identifier for the cop initiating the query. Data automatically deleted within an interval.

      10 replies →

  • mrguyorama  20 days ago

    There is zero "solid plan" you can produce that prevents a popular thing in a democratic country from happening. Like, sure, there are supposed to be some amount of base rules to prevent you from gulaging people as soon as you get a 51% vote share, but if you have enough popularity for long enough, as designed, you can change those rules and eventually do whatever you want.

    You can bet the shit the Nazis did wasn't "allowed" by the Weimar Republic's constitution, but that didn't matter one bit as soon as the brownshirts murdered enough people. Hitler wasn't even that popular at any point. The holocaust didn't happen because Germany didn't have enough "don't do holocausts" rules, it happened because millions of Germans just let it, because they didn't want to die under a brownshirt's boot.

    Meanwhile we've had tens of examples of full blown genocides that did not use any database at all. It has never seemed to actually stop a genocide.

    The answer, as always, is that it takes hard work to defend your rights, and you can never ignore your government, and you should stop trying to ignore your government. You cannot "defang" a government. If enough people are working to build an authoritarian shithole state, they will get it, and no paper will stop them, because "having enough people who want something" is literally what a government is.

    We have thousands and thousands of years of history showing that if you want rights you have to fight for them.

    >If someone points out that the system you're building can be abused

    Any system of authority can be abused. No paper can fix that. The only thing that can fix that is a popular, credible threat to the people trying to abuse it.

int_19h  20 days ago

If you build the system in a way that enables such highly predictable misuse, you do get to share part of the blame.

  • mulmen  20 days ago

    This isn’t even misuse. Sharing with other agencies is an intended feature.

    Edit for clarity this is not a misuse of Flock.

    • ceejayoz  20 days ago

      It's misuse.

      https://oag.ca.gov/system/files/media/2023-dle-06.pdf

      > Importantly, the definition of “public agency” is limited to state or local agencies, including law enforcement agencies, and does not include out-of-state or federal law enforcement agencies. (See Civ. Information Bulletin 2023-DLE-06 California Automated License Plate Reader Data Guidance Page 3 Code, § 1798.90.5, subd. (f).) Accordingly, SB 34 does not permit California LEAs to share ALPR information with private entities or out-of-state or federal agencies, including out-of-state and federal law enforcement agencies. This prohibition applies to ALPR database(s) that LEAs access through private or public vendors who maintain ALPR information collected from multiple databases and/or public agencies.

      1 reply →

roughly  20 days ago

> Don't be angry at the people building them for good.

I am angry because the same people who've argued for years against the kinds of education systems that teach actual social systemic thinking and who've called me naive and cynical for suggesting their pretty toy is going to get people killed are now throwing up their hands and saying "how could we have known?"

Because we fucking told you, that's how.

salawat  20 days ago

Nope. If you're one of them, as a practitioner you should damn well be able to reasonably foresee the pathological use case. Hell, I only cut myself minimal slack for having grown up believing constant exhortations by Oldtimers that "Kid, no one in their right mind would do that," only to see my peer group replacing them do exactly what the Oldtimers were insistent that common sense dictated wouldn't be done.

It is on us to be realistic about how the systems we create will actually be used. I think we lost sight of that in the last couple decades, or figured it wasn't our problem. And the chickens have come home to roost.