Comment by horsawlarway
10 hours ago
At least for your last point... ideally never.
Look, I understand the intent you have, and I also understand the frustration at the lack of care with which many companies have acted with regards to personal data. I get it, I'm also frustrated.
But (it's a big but)...
Your suggestion is that we hold people legally responsible and culpable for losing a confrontation against another motivated, capable, and malicious party.
That's... a seriously, seriously, different standard than holding someone responsible for something like not following best practices, or good policy.
It's the equivalent of killing your general when he loses a battle.
And the problem is that sometimes even good generals lose battles, not because they weren't making an honest effort to win, or being careless, but because they were simply outmatched.
So to be really, really blunt - your proposal basically says that any software company should be legally responsible for not being able to match the resources of a nation-state that might want to compromise their data. That's not good policy, period.
Incidents happen in the meat world too. Engineers follow established standards to prevent them to the best of their ability. If they don't, they are prosecuted. Nobody has ever suggested putting people in jail for Russia using magic to get access to your emails. However, in the real world, there is no magic. The other party "outmatches" you by exploiting typical flaws in software and hardware, or, far more often, in company employees. Software engineering needs to grow up, have real certification and standards bodies and start being rigorously regulated, unless you want to rely on blind hope that your "general" has been putting an "honest effort" and showing basic competence.
We already have similar legal measures in software for following standards. These match very directly to engineering standards in things like construction and architecture. These are clearly understood, ex SOC 2, PCI DSS, GDPR, CCPA, NIST standards, ISO 27001, FISMA... etc... Delve is an example (LITERALLY RIGHT NOW!) of these laws being applied.
What we don't do in engineering is hold the engineer responsible when Russia bombs the bridge.
What you're suggesting is that we hold the software engineer responsible when Russia bombs their software stack (or more realistically, just plants an engineer on the team and leaks security info, like NK has been doing).
Basically - I'm saying you're both wrong about lacking standards, and also suggesting a policy that punishes without regard for circumstance. I'm not saying you're wrong to be mad about general disregard for user data, but I'm saying your "simple and clear" solution is bad.
... something something... for every complex problem there is an answer that is clear, simple, and wrong.
France killed their generals for losing. It was terrible policy then and it's terrible policy now.
We have HIPAA in the US for health care data. There have been no disastrous consequences to holding people and organizations responsible for breaches.
Sure, and in cases of negligence this is fine. The law even explicitly scales the punishment based on perceived negligence and almost always is only prosecuted in cases where the standards expectations aren't followed.
Ex - MMG for 2026 was prosecuted because:
- They failed to notify in response to a breach.
- They failed to complete proper risk analysis as required by HIPAA
They paid 10k in fines.
It wasn't just "They had a data breach" (ops proposal...) it was "They failed to follow standards which led to a data breach where they then acted negligently"
In the same way that we don't punish an architect if their building falls over. We punish them if the building falls over because they failed to follow expected standards.
Buildings don't just fall over, and security breaches don't just happen. These things happen when people fuck up. In the architecture world we hold individuals responsible for not fucking up--not the architect, but instead the licensed engineer who signs and seals the structural aspects of a plan. In the software world we do not.
It's important to occasionally execute or imprison a general to motivate the remaining generals. Rarely though.
> any software company should be legally responsible for not being able to match the resources of a nation-state that might want to compromise their data
No. Not the company, holding companies responsible doesn't do much. The engineer who signed off on the system needs to be held personally liable for its safety. If you're a licensed civil engineer and you sign off on a bridge that collapses, you're liable. That's how the real world works, it should be the same for software.
Define "safety".
Obviously if someone dies or is injured a safety violation has occurred. But other examples include things like data protection failures--if for example your system violates GDPR or similar constraints it is unsafe. If your system accidentally breaks tenancy constraints (sends one user's data to another user) it is unsafe. If your system allows a user to escalate privileges it is unsafe.
These kinds of failures are not inevitable. We can build sociotechnical systems and practices that prevent them, but until we're held liable--until there's sufficient selection pressure to erode the "move fast and break shit" culture--we'll continue to act negligently.
2 replies →