Comment by ckastner
4 years ago
> "The operational semantics of the most basic primitives of your operating system are designed to simplify the implementation of shells."
Yes, but why is this characterized as something negative?
Isn't that the entire point? Operating systems are there to serve user requests, and shells are an interface between user and OS.
Shells simply developed features that users required of them.
> Isn't that the entire point?
The exokernel people would disagree.
You see, an operating system as commonly conceived has at least two major jobs:
- abstract away underlying hardware
- safely multiplex resources
And do the above with as little overhead as possible.
Now the thing is: whenever you have multiple goals, you need to make trade-offs, and you aren't as good at any one goal as you could be.
So the exokernel folks made a suggestion in the 90s: let the OS concentrate on safely multiplexing resources, and do all the abstracting in user level libraries.
See eg https://www.classes.cs.uchicago.edu/archive/2019/winter/3310... or https://people.eecs.berkeley.edu/~kubitron/cs262/handouts/pa...
Normal application programming would mostly look the same as before, your libraries just do more of the heavy lifting. But it's much easier to swap out different libraries than it is to swap out kernel-level functionality.
That vision never caught on with mainstream OSes. But: widespread virtualisation made it possible. You can see hypervisors like Xen as exokernel OSes that do the bare minimum required to safely multiplex, but don't provide (many) abstractions.
Shells have relatively simple operational models, so _any_ API would probably be workable for shells.
Meanwhile, programs with more complex requirements have to work around these APIs. And many programs call other programs, or otherwise have to do tricky process lifecycle management.
The lowest-level APIs should, in theory, cater to the most complex cases, not to the simplest ones. This doesn't prevent a simpler API from existing, but catering to a simple use case in the primitives does hinder more complex needs.
(I think the more nuanced point is that the OS itself might not have a much better design available in any case. Unixes have a lot of neat stuff, but it's a lot of "design by user feature request", and "standardize 4 slightly different ways of doing things", so there is a lot of weirdness and it's hard to have The Perfect API in that case)
> Shells have relatively simple operational models, so _any_ API would probably be workable for shells.
You'd think that, but implementing the UNIX shell and all of its semantics (piping, redirection, waiting, child reaping, jobs, foreground/background, prompting etc) using fork/clone + exec* is way more simple than, say, on Windows. Some API designs are better for that specific task
> Shells have relatively simple operational models, so _any_ API would probably be workable for shells.
True. Today anyways. Back in the 70s though, there was a lot of innovation going on around process spawning, and fork+exec almost certainly made it easy to play with those ideas. I'm referring to job control, for example. But also things like the parent-child relationships between the shell and all the processes in a pipeline -- not all shells have set those up the same way.
So, yeah, maybe we need not just posix_spawn() but posix_pipeline_spawn(), why not. Make it even easier to write a shell. After all, plumbing a complex pipeline with posix_spawn() requires a fair bit of code.
Will any API do? Yes, provided it covers all the things Unix shells do nowadays. It's still easiest to get all the functionality (that a shell dev might want to build) with fork+exec though, especially since the shell author gets a great deal of control that way, though they get that at the price of having to know a great deal of stuff intimately. Arguably, anyone wishing to implement a posix_pipeline_spawn() would be like a shell developer.
The thing is that there are many other programs which require process control, which are not shells. Orders and orders of magnitudes of programs which are not shells. So we can optimize an API for building shells, but it's not going to make writing those other programs easier.
Shells are cool and good, and I don't want to discount fork too much, just saying that the API design space isn't _only_ for shells.
> Yes, but why is this characterized as something negative?
Unfortunately, the text does not provide sufficient context. Shell are not properly supported in any OS (probably except plan9), since 1. the OS provides no enforcement or convention of CLI API interface (there is no enforced encoding standard or checkable stuff), 2. the OS provides no rules for file names to be shell-friendly and 3. there are no dedicated communication channels towards shells or in between programs and shells.
So all in all, shells remain a hack around the system that is "simple to implement the initials" and is annoying to use and write at many corner cases.
> Shells simply developed features that users required of them.
Cross out "simply" and call it convenience+arbitrary complex scripting glue for 4 main goals: 1. piping 2. basic text processing 3. basic job control 4. path hackery
Shells haven't been the primary interface between the user and the OS for decades.
"The primary interface between the user and the OS" is the definition of "shell". That's why the Microsoft Windows process that draws the Start button and filesystem windows is called "the Windows shell".
I don't think OP meant shell as in the Windows shell, or Linux DEs. I mean, how many of those use fork() even on Linux, or would be easier to implement if they did?
3 replies →