Comment by klabb3
14 hours ago
From 20y experience and CS degree, I see software engineering as a constant struggle against accidental complexity. Like quicksand, every movement you make will pull you deeper, even swimming in the right direction. And like entropy, it governs all things (there are no subfields that are free of complexity). It even seems impossible to give a meaningful, useful definition, perhaps by necessity. All is dark.
But now and then, something beautiful happens. Something that used to be dreadful, becomes "solved". Not in the mathematical strict sense, but some abstraction or some tool eliminates an entire class of issues, and once you know it you can barely imagine living without it. That's why I keep coming back to it, I think.
As a species, I think we are in the infancy stages of software engineering, and perhaps CS as well. There's still lots of opportunity to find better abstractions, big & small.
I'm a tech lead and pushing back against accidental complexity is basically my fulltime job.
I'm an Engineering Manager, and I think I have a similar role just applied to people processes rather than code. One nuance though - a lot of the time I suspect it's deliberate complexity designed to obfuscate how little people actually do.
Thats projection.
2 replies →
This was really well written and I agree with you completely. Though I am not so optimistic as a species we have much runway left to get meaningfully much farther out of that infancy.
As tech progresses and those abstractions become substantially more potent, it only amplifies the ability of small groups to use them to massively shape the world to their vision.
On the more benign side of this is just corporate greed and extraordinary amplification of wealth inequality. On the other side is authoritarian governments and extremist groups.
Wealth inequality is a direct cause of authoritarianism and is not benign.
Agree, I probably worded it poorly but I intended more like “seemingly benign” increase in corporate profits.
Perhaps, but generally annoying millions of technology people tends not to end well for firms. Usually the market simply evolves to better match the fiscal conditions.
https://en.wikipedia.org/wiki/Competitive_exclusion_principl...
The Internet itself will likely further fracture into different ecosystems. =3
Can you provide some examples of these beautiful abstractions or tools?
Memory garbage collection, borrow checker, compile-time static typing in dynamic languages (Typescript, Python).
Language specific for JavaScript: Strict comparison operator === that disables type coercion, together with banning ==.
== allows "5" equals 5.
To get perspective(we know what worked), here’s some 50+ years abstractions:
A file is a simple stream of bytes in Unix. (If you think what else it might be then compare to Multics’ segments). Separate processes that may be connected using simple standard I/O streams [pipe] (vs everything is DLL in Multics) — the concept of shell itself (policy vs. mechanism separation http://www.catb.org/esr/writings/taoup/html/ch01s06.html ).
https://retrocomputing.stackexchange.com/questions/15685/wha...
For comparison, you need a new app on iOS for what might have been a shell pipeline (hierarchical file system is absent at user level).
Take message queues. ZMQ and the like have basically solved message passing which was a ghastly thing to worry about for many years.
I disagree. I worked at a protocol designer and implementor for years before people settled on the message queue as the universal abstraction. at the bottom end dumping serialized objects into tcp connections gets you most of the way. and at the top end there is so much leverage around locality, addressing, and transport that we are leaving a lot on the table.
message queues arent at all bad, but they come with additional complexity (most of it operational), and come with a set of limiting assumptions. so my frustration is that they are now the default answer for everything, and we're ignoring this lovely design space, one that becomes increasingly important when talking about scale.
Build tools that enforce hermeticity (cannot depend on files not declared as a dependency) and hashes files (as opposed to using timestamps). This eliminates whole classes of complaints against make.
The actor model for concurrency.
Read The Linux Programming Interface book