Comment by WalterBright
3 years ago
As a former Boeing engineer, other industries can learn a great deal from how airplanes are designed. The Fukushima and Deepwater Horizon disasters were both "zipper" failures that showed little thought was given to "when X fails, then what?"
Note I wrote when X fails, not if X fails. It's a different way of thinking.
When I worked in an industrial context, some coding tasks would seem trivial to today's Joe Random software dev, but we had to be constantly thinking about failure modes: from degraded modes that would keep a plant 100% operative 100% of the time in spite of some component being down, to driving a 10m high oven has the opportunity to break airborne water molecules from mere ambient humidity into hydrogen whose buildups could be dangerously explosive if some parameters were not kept in check, implying that the code/system has to have a number of contingency plans. "Sane default" suddenly has a very tangible meaning.
> we had to be constantly thinking about failure modes
This to me is the biggest difference between writing code for the software industry vs. an industrial industry.
Software is all about the happy path ("move fast and break things") because the consequences typically range from a minor inconvenience to a major financial loss.
Industrial control is all about sad paths ("what happens if someone drives a forklift into your favorite junction box during the most critical, exothermic phase of some reaction") because the consequences usually start at a major financial loss and top out in "Modern Marvels - Engineering Disasters" territory.
You do /not/ want to make it on the USCSB YouTube channel.
Yeah, I work as a Functional Safety Engineer in the process and machinery sector and 90%+ of effort is in planning, considering all the possibilities outside of intended operation and traceability.
I have worked on projects where in retrospect the LOC generated per day, if spread out across the whole project, were between 1 and 3.
But typically, writing of the code does not even commence in the first year, sometimes two.
Then there is the test cases and test coverage etc etc.
This is the difference between engineering code and just producing it - all the effort that goes into understanding all the unwanted code behaviour that may occur and how to detect, manage and/or avoid it.
Implicit state is the enemy, therefore the best code has all states explicitly defined.
As an engineer I think a lot about tradeoffs of cost vs other criteria. There is little I can learn from nuclear or aviation industry, as the cost structure ist so completely different. I’m very happy that the costs of safety in aviation are very good accepted, but I understand that few people are willing to pay similar costs for other things like, say, cars.
The costs of the Fukushima and Deepwater Horizon were very, very high. Both could have been averted at trivial expense with simple changes to the design.
Fukushima:
badthink - the seawall is high enough that it will stop tidal waves
goodthink - what happens when the seawall is overtopped? Answer: the backup generators drown. Solution: put the backup generators on a platform.
Deepwater Horizon:
badthink - the pipe is strong enough to never break
goodthink - what happens when there's enough force to bust the pipe off? Answer: the pipe flow cannot be shut off. Solution: put a fuse (a weak spot) above the valve, so when the pipe busts off, it breaks above the valve, and the valve can be turned to shut off the flow. (The valve was located on the sea floor.)
This is so easy in retrospect when you know what the failure mode will be.
badthink: the Fukushima backup generators must be placed on a platform to keep them out of the range of a once in a millenium tsunami
goodthink: what happens when a typhoon comes and damages the generator on an exposed platform; an event which happens predictably and far more often than tsunamis. Answer: put the backup generators in the basement of a reactor building behind a large seawall. What catastrophe could put the reactor building completely underwater, and still have the reactor survive?
Yeah, trivial changes to the design can prevent all sorts of disasters, but you have to know what you are trying to prevent in a world of infinite complexity
1 reply →
We're making a niche B2B application, and this is very much it for us as well.
Our customers are in a cutthroat market with low margins. We can't spend a ton on pre-analysis, redundancies and so on.
Instead we've focused reduced the impact of failures.
We've made it trivial to switch to an older build in case the new one has an issue. Thus if they hit a bug they can almost always work around it by going to an older build.
This of course requires us to be careful about database changes, but that's relatively easy.
You can not. AI though, can be cheap enough to produce that. I wonder what happens if you take a b2b application and let it rewrite with AI to Nuclear Industry/ Aviation standards into a seperate repo. Then on fixes/rewrite the engineers take the "safety aware repository" as inspiration.
5 replies →
I don't think that's the right way to reason about it.
I find that I can learn a ton from those industries, and as a software engineer I have the added advantage of being able to come up with zero-cost (or low cost), self-documenting abstractions, testing patterns, and ergonomic interfaces that improve the safety of my software.
In software, a lot of safety is embodied in how you structure your interfaces and tests. The biggest cost is your time, but there are economies of scale everywhere. It really pays to think through your interfaces and test plan and systems behavior, and that's where lessons from these other industries can be applied.
So yeah, if you think of these lessons as "do tons of manual QA", you'll run into trouble resourcing it. But you can also think of them as "build systems that continuously self-test, produce telemetry, fail gracefully in legible ways and have multiple redundancies".
Cars might not be the best example, since human lives are at stake, as in aviation. Unless you work on Teslas autopilot, it seems. But yes, backups and restores are often good enough.
As it turns out (and as much as we wouldn’t want them to) human lives are still subject to cost/benefit analysis.
An airliner is a lot of lives, a lot of money, a lot of fuel, and a lot of energy. Which is why a lot has been invested in training, procedure, and safety systems.
Cars operates in an environment which is in most ways a lot more forgiving, they’re controlled by (on average) low-training low-skill non-redundant crews, they’re much more at risk of “enemy action”, the material stresses are in a different realm, and they’re much, much more sensitive to price pressure.
Hell, the difference is already visible in aviation alone, crop dusters and other small planes are a lot less regulated amongst every axis than airliners are.
3 replies →
There are a fair amount of backups in your car. For example, the braking system is dual. There's also engine braking and the parking brake that can be used. All the "energy absorbing" features are a backup for when you crash.
Any substantiation for "Unless you work on Teslas autopilot, it seems"?
I mean you're implying that there are more accidents with autopilot than without it, right? Seems like quite the claim...
6 replies →
In the context of disasters that happened due to software failures (e.g. Ariane 5 [1]), one of my professors used to tell us, that software doesn't break somewhen but is broken from the beginning.
I like the idea of thinking 'when' instead of 'if', but the verdict should be even harder when it comes to software engineering because it has this rare material at its disposal, which doesn't degrade over time.
[1] https://en.wikipedia.org/wiki/Ariane_5#Notable_launches
An example of zipper failure in the Airbus incident is when a wire bundle gets cut, all the functions of all the wires in that bundle are lost. Having two or more smaller bundles physically separated would greatly reduce that risk. Certainly, having the primary and the backup system in the same bundle is a bad idea.
On the 757, one set of control cables runs under the floor. The backup set runs in the ceiling.
It’s the same on Airbus aircraft, I can tell you from experience.
I thought Airbus was fly-by-wire, not cables?
1 reply →
What's fascinating about airplane design for me is not the huge technical complexity, but rather, the way it is designed such that a lot of its subsystems are serviceable by technicians so quickly and reliably, not just in a fully controlled environment like a maintenance hangar, but right on the tarmac, waiting for takeoff.
Designing the airplane to minimize required maintenance and to make maintenance and inspections easier and faster is a huge issue for the engineering department. Also make it very difficult for the mechanics to do things wrongly.
As it was pointed out to me, airplanes sitting on the ground are a black hole sucking up money. Airplanes in the air carrying payload (note the "pay" in payload) are making money. Boeing understands this very well, and is very focused on getting that airplane in the air making money as much as possible.
> When my AoA sensor fails, then what?
crickets, let's just randomise which sensor we use during boot, that ought to do it!
> Airlines really want to be able to use pilots' existing type-rating on this hulking zombie of a 60s-era airframe with modern engines but it behaves differently under certain conditions, what do we do?
let's just build a system that pushes the nose down under those conditions, have it accept potentially unreliable AoA data, and not tell pilots about it!
"AoA sensor" - Angle of Attack sensor.
And the reference is presumably to 737 MAX accident. https://www.afacwa.org/the_inside_story_of_mcas_seattle_time...
Epic fail indeed, costing many lives.
I agree in principle, but I don't think industries should be looking at current-day Boeing's engineering practices except for an example of how a proud company's culture can rot from the inside out with fatal consequences.
I think Boeing has had some difficulties. They have also had some undeniable successes. The 777 and 787 programs have no in-service passenger fatalities attributable to engineering errors to date. That's a monumental achievement.
The 787 has no hull losses at all right? And it’s been flying for 10 years now.
1 reply →
Reminder that this article was about an aircraft built by Airbus.
(Airbus is not Boeing.)
How are aeroplanes designed differently at Boeing vs Airbus? What's the secret sauce?
9 replies →
Are you serious in saying that other industries could learn from Boeing?
Glancing at Walter Bright's brief Wikipedia page - I'd say he worked for Boeing well before they succumbed to the McDonnell Douglas Brain Fungus.
He didn't actually say that.