Comment by jes5199
20 days ago
I’m not sure it still makes sense to do OS research so close to the metal. Most computing is done up on the application level, and our abstractions there suck, and I haven’t seen any evidence that “everything is a file” helps much in a world of web APIs and SQL databases
Some of us are still interested in the world underneath all that web stuff!
Multiple experimental operating systems at multiple abstraction levels sounds like a good idea, though. What sort of system software would you like to build?
I’m actually building an “OS” that’s up a level. it’s more like git, it has a concept of files but they’re documents in a distributed store. I can experiment with interaction patterns without caring about device drivers
Operating systems are where device drivers live. It sounds awfully impractical to develop alternatives at this stage. I think OP is right.
I think OSes should just freeze all their features right now. Does anyone remember all the weird churn in the world of Linux, where (i) KDE changed from version 3 to 4, which broke everyone's KDE completely unnecessarily (ii) GNOME changed from version 2 to 3, which did the same (iii) Ubuntu Linux decided to change their desktop environment away from GNOME for no reason - but then unchanged it a few years later? When all was said and done, nothing substantive really got done.
So stop changing things at the OS level. Only make conservative changes which don't break the APIs and UIs. Time to feature-freeze, and work on the layers above. If the upper layers take over the work of the lower layers, then over time the lower layers can get silently replaced.
> Ubuntu Linux decided to change their desktop environment away from GNOME for no reason
Oh, there absolutely were reasons. I covered them here:
https://www.theregister.com/Print/2013/06/03/thank_microsoft...
I have never had so much negative feedback and ad-hom attacks on HN as for that story, I think. :-D
Short version, the chronology goes like this:
2004: Ubuntu does the first more-or-less consumer-quality desktop Linux that is 100% free of charge. No paid version. It uses the current best of breed FOSS components and they choose GNOME 2, Mozilla, and OpenOffice.
By 2006 Ubuntu 6.06 "Dapper Drake" comes out, the first LTS. It is catching on a bit.
Fedora Core 6 and RHEL 4 are also getting established, and both use GNOME 2. Every major distro offers GNOME 2, even KDE-centric ones like SUSE. Paid distros like Mandriva and SUSE as starting to get in some trouble -- why pay when Ubuntu does the job?
Even Solaris uses GNOME 2.
2006-2007, MS is getting worried and starts talking about suing. It doesn't know who yet so it just starts saying intentionally not-vague-at-all things like the Linux desktop infringes "about 265 patents".
This is visibly true if you are 35-40 years old: if you remember desktop GUI OSes before 1995, they were all over the place. Most had desktop drive icons. Most had a global menu bar at the top. This is because most copied MacOS. Windows was an ugly mess and only lunatics copied that. (Enter the Open Group with Motif.)
But then came Win95. Huge hit.
After 1995, every GUI gets a task bar, it gets buttons for apps, even window managers like Fvwm95 and soon after IceWM. QNX Neutrino looks like it. OS/2 Warp 4 looks like it. Everyone copies it.
Around the time NT 4 is out and Win98 is taking shape, both KDE and GNOME get going and copy the Win9x look and feel. Xfce dumps its CDE look and feel, goes FOSS, and becomes a Win95 copy.
MS had a case. Everyone had copied them. MS is not stupid and it's been sued lots of times. You betcha it patented everything and kept the receipts. The only problem it has is: who does it sue?
RH says no. GNOME 3 says "oh noes our industry leading GU is, er, yeah, stale, it's stagnant, it's not changing, so what we're gonna do is rip it up and start again! With no taskbar and no hierarchical start menu and no menu bars in windows and no OK and CANCEL buttons at the bottom" and all the other things that they can identify that are from Win9x.
GNOME is mainly sponsored by Red Hat.
Canonical tries to get involved; RH says fsck off. It can't use KDE, that's visibly a ripoff. Ditto Xfce, Enlightenment, etc. LXDE doesn't exist yet.
So it does its own thing based on the Netbook Launcher. If it daren't imitate Windows then what's the leading other candidate? This Mac OS X thing is taking off. It has borrowed some stuff from Windows like Cmd+Tab and Fast User Switching and stuff and got away with it. Let's do that, then.
SUSE just wearily says "OK, how much? Where do we sign?"
4 replies →
That's why it's a research OS, a lot of people (or at least some) think that the current range of mainstream OS are not very well designed, and we can do better.
I'm not saying Plan 9 is the alternative, but it is kind of amazing how un-networked modern Operating Systems are, and we just rely on disparate apps and protocols to make it feel like the OS is integrated into networks, but they only semi-are.
I didn’t really see the appeal until I learned how to use FUSE.
There’s something elegant about filesystems. Even more than pipes, filesystems can be used to glue programs together. Want to control your webcam with Vim? Expose a writable file. Want to share a device across the network? Expose it as a file system, mount that filesystem on your computer.
Idk I still find low level OS stuff super interesting because it hasn't had a rework in so long. With everything we've learnt since the age of modern computing, drives larger than a few MBs, super fast memory and fast cryptography to name a few.
It's interesting to imagine a new OS that incorporates these changes from it's infancy.
I appreciate all of the effort put in by Linux, BSD, Android, QNX and closed source OSs' have put in to building upon existing ideas and innovating gradually on them. But man I really want to see something better than everything is a file. I really enjoyed the stuff BeOS was pitching.
Well, on the file system side BeOS was pitching "virtual folders" that are really no different than what plan9 provides.
The most "research" thing I'm aware of 9front (since you're speaking in present tense) doing is that GEFS needed to work out a lot of things for itself that weren't in the Bε-tree proof-of-concept FS that came before.
I dunno how "close to the metal" you'd consider that.
("GEFS" being a disk filesystem that's been discussed on HN.)
The "everything is a file" approach is nice in many cases, I'm worried though if it works everywhere. Maybe if done right. Subversion (SVN) shows branches as separate file trees.. and ClearCase too (though I'm on thin ice with ClearCase, having used it very little). And I just can't stand the file-oriented way SVN works, I could never get used to it. But there are a lot of other cases where "it's a file" does work, I've experimented with creating Fuse filesystem interfaces to some stuff now and then.
> I haven’t seen any evidence that “everything is a file” helps much in a world of web APIs and SQL databases
Well for one thing, such an abstraction enables you to avoid web apis and sql databases!
You're going to have to explain to me how a parametrized request/response system like calling a Web API or making a SQL query can be mapped to reading files. I've seen some stuff that people do with FUSE and it looks like ridiculous circus hoop jumping to making the Brainfuck-is-Turing-complete version of a query system. We have syntax for a reason.
Typically, if you were writing your hypothetical sql client in rc shell, you'd implement an interface that looks something like:
This is also roughly how webfs works. Making network connections from the shell follows the same pattern. So, for that matter, does making network connections from C, just the file descriptor management is in C.
5 replies →
Plan9 allows for implementing file servers in user space and exporting a whole file tree as a virtual "folder", so it's really more of "everything as a file server". No different than FUSE, really.
7 replies →
In addition to the sibling comment, you might also consider simply not using the APIs or SQL queries to begin with. Many people have entire careers without touching either.
6 replies →
Can Plan 9 do transactions? If so, it is unsuitable for being a database. It can run databases, because those can work without transactions. But can't do native writes without them. Can it do transactional reads? How would you represent isolation levels?
How do you do a join on Plan 9? I get the impression that these are coded in each client. But complicated queries need indexes and optimizer. SQL database has advantage that can feed it and it figures out the plan.
Plan 9 is just a brand smeared across a codebase, just like every other operating system.
> If so, it is unsuitable for being a database. It can run databases, because those can work without transactions. But can't do native writes without them. Can it do transactional reads? How would you represent isolation levels?
Bruh ask 9front
Indeed, no, we shouldn't be sure everything-is-a-file makes sense to do OS research. I don't think this is particularly necessarily what need to considered close to the metal. But it is os research.
I think you're right about where computing is today. It's mostly at the app level.
I think you once again hit a super hard conventionality chord & speak to where we are by saying we don't have much evidence of "everything is a file* helping, anywhere. Broadly.
But analyzing where we are & assessing they everything-is-a-file isn't a sure thing doesn't dissuade me. Apps have wanted control, and there's beenfew drivers to try to unite & tie together computing. App makers would actively resist if not drag their feet against giving up total dominion of the user experience. OS makers don't have the capital to take over the power from apps. The strain of unweaving this corporate power interests is immense.
There have been some attempts. BeOS tried to do interesting things with enriching files, with making their more of a database. Microsoft's cancelled WinFS is rumored to have similarly made a sort of OS filesystem/database hybrid that would be useful to the users without the apps. But these are some of the few examples we have of trying anything.
We're in this era where agents are happening, and it's clear that there's very few clear good paths available to us now for agents to actuate & articulate the changes they could and should be doing. Which is just a reflection of app design where the system state is all bundled up deeply inside these bespoke awkward UIs. App design doesn't afford good access, and part of the proof is that other machines can't control apps short of enormous visual processing, which leaves much ambiguity. If agents can't it also strongly implies humans had little chance to master and advance their experience too.
I strongly think we should have some frontiers for active OS research that are user impactful. We ought be figuring out how to allow better for users, in ways that will work broadly & cross cuttingly. Everything is a file seems like one very strong candidate here, for liberating some of the power out of the narrow & super specific rigid & closed application layer.
I think Dan was also super on point writing A Social Filesystem. Which is that social networks & many online systems are everything-as-a-file under the hood. And that there is generic networked multi-party social networking platform available, that we have a super OS already here that does files super interestingly. And Dan points out how it unlocks things, how not having one specific app but having our online data allow multiple consumers, multiple tools, is super interesting an opening.
So, everything is a file is very webful. A URL logically ought be. A multi-media personal data server for every file you can imagine creates an interest powerful OS, and a networked OS.
And users have been warped into fitting the small box their apps demand of them so far. They've had no option about it. All incentive has been to trap users more and more to have no off roads to keep your tool being the one tool for the job.
Distribute the power. Decentralize off the app. Allow other tools. Empower broader OS or platform to let users work across media types and to combine multiple tools and views in their workflow. Allow them to script and control the world around them, to #m2m orchestrate & drive tool use.
I don't disagree with anything you said I just think it's a 30 year old basis you stand from, one that hasn't helped had gotten better and which has ongoingly shrunk what is possible & limited the ability to even start trying for more or better. I don't think we are served by what it feels like you are trying to highlight. And I think "everything is a file" could be an incredible way to start opening up better, possibly, maybe!! but I'm very down to hear other reasonable or out there ideas!! I'm just not interested in staying in the disgraceful anti-user app-controlled unyielding quagmire we have been trapped in for decades.
I guess I feel like if we’re rewriting device drivers then we’re in a turing tarpit. I think there’s room for innovation at what is traditionally considered the application level - we run git, postgres, document stores etc as applications. I think the way to solve the next generation of coordinating is by doing more interesting stuff on this layer.