Comment by mehrdadn
7 years ago
100% agree with that takeaway in general, except that in this case I think POSIX does actually say this. I pointed out and discussed this a few months ago [1] (and it's already discussed on the page as well), but basically, those who disagree believe that fsync's task is to merely "send" the data to the storage device, not actually make sure it's written persistently... which I see as being blatantly inconsistent with "to assure that after a system crash or other failure that all data up to the time of the fsync() call is recorded to the disk". There's nothing on the page that says "data since the last call to fsync", and that sentence says that was definitely not the intention, and yet somehow that's how people read it to support the notion that current implementations are correct (or vice-versa: they use current implementations as evidence that this is the correct reading).
Perhaps? The people in the thread seem fairly convinced that's not what it says. When I read it, I see "... all data for the open file descriptor named by fildes is to be transferred to the storage device ..." So, in the event of a failure in the underlying hardware, the data may have been transferred, but subsequently an error has occurred. I don't see how the sentence obliges the implementation to transfer multiple times. I'm not an expert though.
In any case, the problem is with the attitude and how opinions are expressed, not the exact state of which opinion is being expressed. :) Also, it is plainly not "entirely clear" since we two apparently reasonable people disagree on what the sentence means. So even if the person is right about what was ultimately intended when the spec was written, they are still incorrect in their view that it's clear what POSIX says.
That sentence's language can be improved then if a sufficiently determined kernel developer (not sure I'd say "reasonable" but whatever) can misread that sentence in that way (while ignoring the entire "RATIONALE" section). But what value is a guarantee that it's transferred to the storage device and not that it's actually committed to permanent storage? If someone's best argument is that they correctly provide a completely useless guarantee and that there's nothing that provides the guarantee that people actually need for real work, they're on pretty shaky footing.
Presumably people will not buy permanent storage that does not commit data that is transferred to it? The thing is, the fsync guarantee isn't useless. You "just" have to rewrite any data that you had previously written since the last sync, and be tolerant of partially committed data in the interim.
2 replies →
> So, in the event of a failure in the underlying hardware, the data may have been transferred, but subsequently an error has occurred. I don't see how the sentence obliges the implementation to transfer multiple times. I'm not an expert though.
That is actually an issue with disks which have a small memory buffer before actually writing to persistent storage: not only will they acknowledge success as soon as the data's in their buffer (losing information in case of error in the underlying storage write) they may not properly handle a power loss of some sort, ack'ing the write to the buffer then losing it before it has the time to hit storage.