Comment by brookst

2 months ago

Can we retire the “seatbelts are useless because they can’t prevent every loss of life” approach to risk mitigation please?

If the acceptance criteria is “would prevent every single past instance and every imaginable future instance”, then yes, no mitigation is every sufficient to address any problem in the world, so we might as well give up.

But I don’t think that’s the right lens to use.

5 comments

brookst

Reply
pjc50  2 months ago

That depends on whether it's a issue of accidents or a "you have to get lucky every time, we only have to get lucky once" issue.

  • brookst  2 months ago

    Death only has to get lucky once. Are you going to stop wearing seatbelts?

    • ben_w  2 months ago

      I assume pjc50's quotation is referencing a quote attributed to a terrorist group after they failed to assassinate the UK Prime Minister: https://quoteinvestigator.com/2025/12/08/lucky-always/

      You're in control of how much danger of accident you expose yourself to.

      Nobody is in control of how much danger we are exposed to from other people who are actively trying to do us harm, who will keep going until they get what they're after or are stopped.

      For most people, seatbelts are the former. Yeah, not perfect, but they reduce risk. For the latter, if you're known to be a seatbelt wearer, the attacker just does something where seatbelts don't matter.

      Every new AI model introduces new capabilities and competencies, so we're not even sure what the true risk levels are yet for self-exposure in this category. The restrictions on AI may be like seatbelts and speed limits, or they may be like "if you install a 1000 HP turbojet engine in your Honda Civic it will no longer be road legal". And this analogy also includes how the first cars had speed limits set low enough to not risk the horse industry, i.e. we may be too cautious.

      1 reply →