Comment by 3pt14159
4 years ago
I've worked as a structural engineer (EIT) on bridges and buildings in Canada before getting bored and moving back into software.
There are major differences in designing bridges and in crafting code. So many, in fact it is difficult to even know where to start. But with that proviso, I think the concept of safety versus the concept of security is one that so many people conflate. We design bridges to be safe against the elements. Sure, there are 1000 year storms but we know what we're designing for and it is fundamentally an economic activity. We design these things to fail at some regularity because to do otherwise would require an over-investment of resources.
Security isn't like safety because the attack scales up with the value of compromising the target. For example, when someone starts a new social network and hashes passwords the strength of their algorithm may be just fine, but once they have millions of users it may become worthwhile for attackers to invest in rainbow tables or other means to thwart their salted hash.
Security is an arms race. That's why we're having so much trouble securing these systems. A flood doesn't care how strong your bridge is, or where it is most vulnerable.
Aside: The distinction between safety and security I know:
- safety is "the system cannot harm the environment"
- security is the inverse: "the environment cannot harm the system"
To me, your distinction has to do with the particular attacker model - both sides are security (under these definitions).
That's an interesting distinction, but I think GP meant something else - and I'm willing to agree with their view:
- Safety is a PvE game[0] - your system gets "attacked" by non-sentient factors, like weather, animals, or people having an accident. The strength of an attack can be estimated as a distribution, and that estimate remains fixed (or at least changes predictably) over time. Floods don't get monotonically stronger over the years[1], animals don't grow razor-sharp titanium teeth, accidents don't become more peculiar over time.
- Security is a PvP game - your system is being attacked by other sentient beings, capable of both carefully planning and making decisions on the fly. The strength of the attack is unbounded, and roughly proportional to how much the attacker could gain from breaching your system. The set of attackers, the revenue[2] from an attack, the cost of performing it - all change over time, and you don't control it.
These two types of threats call for a completely different approach.
Most physical engineering systems are predominantly concerned with safety - with PvE scenarios. Most software systems connected to the Internet are primarily concerned with security - PvP. A PvE scenario in software engineering is ensuring your intern can't accidentally delete the production database, or that you don't get state-changing API requests indexed by web crawlers, or that an operator clicking the mouse wrong won't irradiate their patient.
--
[0] - PvE = "Player vs Environment"; PvP = "Player vs Player".
[1] - Climate change notwithstanding; see: estimate changing predictably.
[2] - Broadly understood. It may not be about the money, but it can be still easily approximated in dollars.
I wonder how this distinction plays out in languages that use the same word for safety and security, e.g. German and Portuguese.
You would use "protection" (Schutz) to make this distinction. Also German verbs can have many suffixes, which often help with the direction of an action and thereby changing the meaning (e.g. sichern, absichern, besichern, versichern).
1 reply →
So it's like building a bridge... that needs to constantly withstand thousands of anonymous, usually untraceable, and always evolving terrorist attacks.
...in which the attackers have free access to copies of the bridge where they can silently test attack strategies millions of times per second for months or years on end.
The safety vs security distinction made above is fundamental. Developers are faced with solving an entire class of problems that is barely addressed by the rest of the engineering disciplines.
> where they can silently test attack strategies millions of times per second for months or years on end
Remotely, anonymously, at virtually no risk to themselves.
1 reply →
But its also a case where "perfect" exists. A case where you can, in principle, have perfect information about the internals of your bridge at any point. A case where you can, in theory, design the bridge to handle an infinite load from above.
In software, you can spec the behavior of your program. And then it is possible to code to that exact spec. It is also possible, with encryption and stuff, to write specs that are safe even when malicious parties have control over certain parts.
This is not to say that writing such specs is easy, nor that coding to an exact spec is easy. Heck, I would even doubt that it is possible to do either thing consistently. My point is, the challenge is a lot harder. But the tools available are a lot stronger.
Its not a lost cause just because the challenge is so much harder.
That kind of perfect is possible in math but not in software, which runs on physical machines and was written and verified by humans. It's like building your bridge inside a vacuum chamber with no entrances or exits—possible but not practical.
Or climate change.
I agree that safety & security are frequently conflated, but I don't think the important aspect is that there's no analogy between IT & construction.
IT safety = construction safety. What kind of cracks/bumps does your bridge/building have, can it handle increase in car volume over time, lots of new appliances put extra load on the foundation etc. IT safety is very similar in that way.
IT security = physical infrastructure security. Is your construction safe from active malicious attacks/vandalism? Generally we give up on vandalism from a physical security perspective in cities - spray paint tagging is pretty much everywhere. Similarly, crime is generally a problem that's not solvable & we try to manage. There's also large scale terrorist attacks that can & do happen from time to time.
There are of course many nuanced differences because no analogy is perfect, but I think the main tangible difference is that one is in the physical space while the other is in the virtual space. Virtual space doesn't operate the same way because the limits are different. Attackers can easily maintain anonymity, attackers can replicate an attack easily without additional effort/cost on their part, attackers can purchase "blueprints" for an attack that are basically the same thing as the attack itself, attacks can be carried out at a distance, & there are many strong financial motives for carrying out the attack. The financial motive is particularly important because it funds the every growing arms raise between offence & defense. In the physical space this kind of race is only visible in nation states whereas in the virtual space both nation states & private actors participate in this race.
Similarly, that's why IT development is a bit different from construction. Changing a blueprint in virtual space is nearly identical from changing the actual "building" itself & the cost is several orders of magnitude lower than it would be in physical space. Larger software projects are cheaper because we can build reusable components that have tests that ensure certain behaviors of the code & then we rerun them in various environments to make sure our assumptions still hold. We can also more easily simulate behavior in the real world before we actually ship to production. In the physical space you have to do that testing upfront to qualify a part. Then if you need a new part, you're sharing less of the design whereas in virtual space you can share largely the same design (or even the exact same design) across very different environments. & there's no simulation - you build & patch, but you generally don't change your foundation once you've built half the building.