Comment by nickpsecurity
10 years ago
Great write-up and probably explains some issues in my apps a while back. Like that my long-time favorite, XFS, totally kicks ass in the comparisons. I agree on using static analysis and other techniques as I regularly push that here. What threw me is this line:
"that when they came up with threads, locks, and conditional variables at PARC, they thought that they were creating a programming model that anyone could use, but that there’s now decades of evidence that they were wrong. We’ve accumulated a lot of evidence that humans are very bad at reasoning at these kinds of problems, which are very similar to the problems you have when writing correct code to interact with current filesystems."
There were quite a few ways of organizing systems, including objects and functions, that the old guard came up with. UNIX's popularity and organization style of some others pushed the file-oriented approach from mainstream into outright dominance. However, many of us long argued it was a bad idea and alternatives exist that just need work on the implementation side. We actually saw some of those old ideas put into practice in data warehousing, NoSQL, "the cloud," and so on. Just gotta do more as we don't have a technical reason for dealing with the non-sense in the write-up: just avoiding getting our hands dirty with the replacements.
I think you'll find this an interesting read then: http://teh.entar.net/~nick/mail/why-reiserfs-is-teh-sukc
It's written in 2004 so I don't know how current it is, but it makes the point that XFS makes certain performance & safety guarantees essentially assuming that you're running on hardware that has a UPS with the ability to interrupt the OS saying "oops, we're out of power".
It was designed by SGI for high-end workstations and supercomputers with long-running jobs (esp render farms). So, that doesn't surprise me. However, it's nice to have all the assumptions in the open and preferrably in user/admin guides. Another issue was it zeroing out stuff on occasion but they fixed that.
2004 is not current for XFS, that is a decade ago! However, disks finishing writes and not lying about having done it is a critical need for all FS. For some like ext3 you would notice it less as it was flush happy.
XFS is becoming the sane default filesystem for servers as it allocates nodes more consistently than the other current mainstream linux options on multidisk systems. Basically small servers now have more disk space and performance than the large systems of 2004. So XFS stayed put in where it starts to make sense, but systems grew to meet its sweetspot much often.
Pretty good analysis.