Comment by chubot
2 years ago
FWIW this is highly related to what I think of as the "narrow waist" architectural principle. You may like these articles I wrote:
The Internet Was Designed With a Narrow Waist - https://www.oilshell.org/blog/2022/02/diagrams.html
The Internet has huge utility because TCP/IP is a narrow waist. It has architectural connections to Ethernet/wireless/... on one end, and HTTP/BitTorrent/... on the other.
For the most functionality, you want fewer code components, and more interoperability. You want O(M + N) pieces of code to give O(M * N) functionality.
The narrow waist architecture does that -- it gives O(M * N) connections to O(M + N) amounts of code.
You don't want to write O(M * N) code, though many people and systems are stuck there!
This generally works the best when the connections are data-oriented and protocol-oriented, not oriented around source code.
---
A Sketch of the Biggest Idea in Software Architecture - https://www.oilshell.org/blog/2022/03/backlog-arch.html
I mention Metcalfe's law, which is related but distinct. Metcalfe's law is about O(N^2) network node connections ("dynamically"), while the (usually) O(M * N) narrow waist is about system architecture ("statically").
They both produce network effects! If the Internet already exists, then the easiest design to implement is to attach your new network to it (e.g. a network in space), not create a new, incompatible network.
---
Incidentally, this seems like what's wrong with the Kubernetes ecosystem -- it has an combinatorial explosion of code due to lack of protocols and interoperability.
It's not data-oriented, like Unix is.
(An important point is that lots of people complain about Unix-style unstructured byte streams because it's suboptimal LOCALLY, while missing the global interoperability / scale / system economy issues -- they get stuck writing O(M * N) code to avoid parsing and serializing )
---
Newer article from last year - Oils Is Exterior-First (Code, Text, and Structured Data) - https://www.oilshell.org/blog/2023/06/ysh-design.html
Software modularity within a process is much different than modularity between processes -- it's a bit more like networking.
You need an exterior narrow waist. To pick an example, a consequence of that is that encodings like UTF-16 don't make sense in any channels where you don't have metadata, and there are a lot of those
e.g. the URL comes BEFORE the Content-Type header in HTTP!
Yeah, I can see the similarity. Thanks for the links.
Also, vaguely related, and a way to achieve this goal of making things composable, is the "everything is an X" pattern: https://lukeplant.me.uk/blog/posts/everything-is-an-x-patter...
Yup, the "Everything is an X" link is in the appendix to the second post :)
https://www.oilshell.org/blog/2022/03/backlog-arch.html#wiki...
I call it the Perlis-Thompson Principle, after Perlis:
It's better to have 100 functions on 1 data structure than 10 functions and 10 data structures.
and Thompson:
A program is generally exponentially complicated by the number of notions that it invents for itself. To reduce this complication to a minimum, you have to make the number of notions zero or one, which are two numbers that can be raised to any power without disturbing this concept. Since you cannot achieve much with zero notions, it is my belief that you should base systems on a single notion.
https://www.oilshell.org/blog/2021/08/history-trivia.html