Comment by jasonhansel
3 years ago
I don't think replacing the root FS with a database is a good idea. There are two options here:
1. Make it a database with schemas. In that case, you'll never be able to change or extend the language used to define schemas, since every application and OS will need to keep supporting older versions. It will also be hard to have multiple competing implementations of a complete database system, particularly because applications will come to rely on the performance characteristics of the reference implementation.
2. Make it schemaless, like a key-value store. That basically just is the file system, but non-heirarchical and with a faster implementation. In that case, your DB could just be a faster implementation of the existing filesystem interface or a natural extension thereof. The hierarchical nature of the filesystem is fairly essential if you want to allow multiple applications to avoid trampling on each other.
This pretty succinctly captures it.
A file system is, at its heart, a naming system. Given this name, return me a handle to "do things" with that object. In SQL it is "SELECT * FROM FILES WHERE ( NAME == <name> );". The next debate is what the schema for the files table?
Well known things that would appear in that schema are things like "access rights", "ownership", "access times", Etc. Additionally related things like the "consuming application" might be there, and the "editing application" Etc.
If these things were all relations/schema then a lot of "warts" like MAGICNUMBER or extension type, security certificates, integrity digests, OS requirement, symbol tables, or character encoding can then become schema elements and the names get "purified" (scare quotes because architecture astronauts really get all hot and bothered by overloading things with multiple semantics.
Then you can do "views" like when someone types a name at the shell you can select from files that are executable on this OS and match the names.
There are basically a whole bunch of things that are "grafted on" to the file system and they can (and do) get out of date relative to the files and cause problems. (the canonical cases are moving an executable from one directory to another making it no longer visible for execution, changing the extension or magic number resulting in the wrong thing being used to try to run/edit it)
The architecture reasoning goes, "If there was just one source of truth about these things, a database, then a whole bunch of bugs and user annoyances would vanish."
Anyway, I've witnessed people start down this grand vision, devolve into your #2 above (key/value store) and then finally throw their hands up in surrender.
One of the systems questions I ponder sometimes is at what point is memory so cheap and plentiful that replacing the buffer cache in the kernel with a giant interconnected schema like this gives equivalent "time to record" values. An RDMA accessible victim cache for "blob" buffering might help too.
A couple of people have taken runs at it as "object storage" or "storage as a service" but it isn't quite there yet.
Still, for a complex file format (say MPEG 4) having it be a data base gives you some advantages and makes writing file component parsers unnecessary. So that's a win.