Comment by creer
2 years ago
> It leads to buildings falling down or burning up [with people in them]. This was a common occurrence 100+ years ago. You know what made it less common? Standardization. Building codes. Minimum standards for engineers and the trades.
To me, this is a more interesting comparison. Is it PE certification and contractor licenses that led to this or is it building codes, construction inspectors, occupancy permits? I will argue that it's inspectors, NOT PE or contractors. And I will argue that the buildings codes have major negative consequences also. We all know of constructions methods that would have great benefits but have to be abandonned because they don't easily fit the current code. We all know of buildings that are to-code and yet ridiculously noisy and cheaply built.
I will also argue that there are building code equivalents already in software and system architecture. There are several for "certifying" system or site security and systems that host credit card payments. And we all know how well they work. So I agree with you that there is room for progress there, but I will also argue that the approach NEEDS to be different. The current security or payment checklists are bureaucratic, CYA nonsense which discourage thinking and encourage bureaucracy and CYA specifically in place of actual security. The only thinking they encourage is creative writing to twist reality into the proper buzzwords.
There may be a way to specify practices and security but we sure have not discovered it yet. So, a research question rather than already a standardization question? I will point out also that there WERE directions that did work in the past. For example, Dan Farmer and Wietse Venema's SATAN (and the several descendants since then) was bureaucracy-free: the test showed specific rubber-meets-the-road issues with your system that you could either fix or defend. No bullshit about using a firewall(tm) "because that's best practice".
I also don't say that it's bad to publish books. I will say that it is bad to push "best practice". "Best practice" is precisely bureaucracy and CYA in place of thinking. To the point of site owners defending their lapses in the name of "best practices".
What else currently goes in the right direction? Pen testing. Bug rewards. Code reviews.
You really need both. Mandatory education, degrees, apprenticeships, licenses, etc is how you make sure they know how to do the thing. And then the building codes and inspections is how you check that they did the thing. If you ask someone to build a home "to code" but you never teach them how, they will spend years trying to figure it out, inconsistently. Send them to school, have them apprentice, and afterward they will be able to build it in a month, in a standard way.
You remind me, there is an industry that has some basic software building codes: the Defense Industry. There are some pretty thorough standards for IT components, processes, etc needed to work with the military (even in the cloud). But it is all self-attested, so it's like asking a building contractor to make sure they inspect themselves. Government keeps asking the tech industry to solve this, but nobody wants to take responsibility. As more and more stuff falls apart (in the public & private sector) the government is gonna get louder and louder about this. It's already started with privacy & competition, but big failures like Crowdstrike make it obvious that the rot goes deeper.
I agree that the US defense sector is an excellent example of the kind of credentialism in software that you, and the IEEE, are advocating! And the results are dismaying. As Anduril says in https://www.rebootingthearsenal.com/:
> Despite spending more money than ever on defense, our military technology stays the same. There is more AI in a Tesla than in any U.S. military vehicle; better computer vision in your Snapchat app than in any system the Department of Defense owns; and, until 2019, the United States' nuclear arsenal operated off floppy disks. (...) today, in almost every wargame the United States Department of Defense models against China, China wins.
Of course the DoD's problems go much deeper than just credentialism, but credentialism is definitely one of the causes of the disease, not a palliative measure.
> Is it PE certification and contractor licenses that led to this or is it building codes, construction inspectors, occupancy permits? I will argue that it's inspectors, NOT PE or contractors.
100%