Comment by bombcar
2 years ago
It's kind of a combination of all three.
He was "right place/right time" with a working journalling filesystem for a rapidly expanding Linux world that needed one, desperately. Drive sizes were climbing incredibly fast, and FSCK times were out of control (back then, almost all distros wouldn't mount "dirty" ext2 partitions without a full FSCK, which could take ages on a 100 GB drive.
Some of the concepts of (as he says in the letter, read it) about namespaces were quite up and coming (WinFS had similar concepts, mind you). Other aspects about journaling were also "ahead of the time" but part of the reason he got their first was not making data integrity the number 1 feature (which you can still do with modern filesystems, but Reiser3 pioneered the "journal metadata only" which makes FSCK fast, and access fast, but can result in corrupt data. Most other filesystems default to slow and correct, with an option to go fast if you want.
XFS was similar, but things like ZFS are more powerful in almost orthogonal ways.
His "dream" of the one true storage that melds RAM, ROM, disk, cache, etc has yet to come to be, though you could argue that some of the S3 style interfaces are closer.
Re: “Reiser3 pioneered journal metadata only”, a metadata-only approach to journaling actually was not uncommon (although for Linux, journaling at all for a time was not available).
This first journalled Unix file system in AIX in the early 90s was metadata only, and faster fsck made it an appealing feature copied by all other commercial Unixes by the late 90s. In the latter 90s, DigitalUnix was the first Unix vendor to provide journaling data, not just metadata. Even SGI’s XFS which had a number of cool and unique features around extent structures optimized for serving video stream large file did not have journaled data.
Most people in Unix then (and now) are unaware of “uncool” operating systems like OS/400 (now iSeries) which arguably delivered on that dream of a melding RAM, ROM, disk, cache. It wasn’t POSIX/Unix/hacker-friendly/cheap but it was and I think still is a billion dollar minicomputer business for IBM. I believe they called it “Single-Level Store Architecture”.
Wikipedia for this for the interested (https://en.wikipedia.org/wiki/IBM_i ): “Instead of operating on memory addresses, TIMI instructions operate on objects. All data in IBM i, such as data files, source code, programs and regions of allocated memory, are encapsulated inside objects managed by the operating system (c.f. the "Everything is a file" model in Unix). …The object model hides whether data is stored in primary, or secondary storage. Instead, the operating system automatically handles the process of retrieving and then storing the changes to permanent storage.”
I believe lisp machines back in the day had something grossly similar.
The difficulty (impossibility?) of implementing this sort of file system in the confines of POSIX (with extensions even?) is probably why this never happened in Unix.
As Reiner’s 2023 letter we are discussing notes, this required new information/hints flowing up and down the layers of the stack. It’d require implementing a new userspace+kernel paradigm of application development alongside the current one.
Accomplishing that sort of transformation in an ecosystem takes a very unique blend of skills and knowledge, and it’s additionally not clear how much economic benefit accrues to anyone for making this shift which is an inhibitor to realization, in Unix, of “the dream”.
But it was done outside of Unix and it quietly is running Fortune 100 workloads today … although many of them are implementing the “strangler pattern” to get those workloads moved to more conventional “modern” systems.