← Back to context

Comment by EugeneOZ

4 years ago

Add a secondary UPS. What will be the next excuse?

It's ridiculous to think that in case of power loss you expect 100% data integrity - it might happen in the middle of the command execution. If the system should be unkillable, it should have an unkillable power source in the first place.

21 comments

EugeneOZ

Reply
marcan_42  4 years ago

The entire point of modern journaling filesystems and properly designed transactional databases is to ensure 100% data integrity in case of power loss, every time, no matter what. The thinking you have is from the 1990s. We can (and do) do better today.

A properly designed transactional database will only ever "fail ahead". If power fails a transaction that was in the process of committing might commit without an ack, but will never return an ack and then be lost on the next startup. The ack means the data is safe, regardless of what happened afterwards.

  • EugeneOZ  4 years ago

    After reading this comment I’m not so sure about that: https://news.ycombinator.com/item?id=30371181

    • marcan_42  4 years ago

      That comment is about the semantics of OS APIs; filesystems are designed not to corrupt themselves in case of hard shutdown, and this is true as long as the underlying storage is well-behaved (e.g. honors flush requests). Databases on macOS already use F_FULLFSYNC (if they noticed this issue) to provide those guarantees. On Linux they just use fsync().

coldtea  4 years ago

>Add a secondary UPS. What will be the next excuse?

You shouldn't have to add a secondary UPS at all, period, and still get that.

Databases are designed that way (for integrity under sudden power loss) - the OS just needs to provide a standard call for the sync that they can use.

Now, fsync not guaranteeing a write is one thing -- and it's common in other OSes, even Linux behaved like that.

The non-commital fullsync on the other hand (and the slow speed) are problematic, and that's not an excuse for the user having such a bizarro case as wanting to run a DB on their Mac Mini without 2 UPS, that's you excusing Apple.

Not to mention that 2 UPS wont solve the problem if you're not there to shut down the computer gracefully as they, themselves, are depleted (e.g. at night) when there's a powerloss.

freemint  4 years ago

Since no Mac device has two power support adding a second UPS means chaining which will only increase the risk of something going wrong in the chain.

Nobody expects 100% data integrity on power of. What is expected is that data that what was fsynced has 100% data integrity once that system call returns. This information is also used when moving files across the network, the file gets deleted on the sender when the receiver said fsync is completed. This means you could loose entire files of data when moving things over the network onto a Mac

  • marcan_42  4 years ago

    For what it's worth, adding an external UPS to a Mac laptop counts as two, and in fact you can add one per Type C port + MagSafe, so you can have up to 5 battery backups for the 2021 Macbook Pro line (internal + MagSafe + 3 x Type C).

    It's silly, but you can :-)

    • freemint  4 years ago

      I never considered Mac Laptops as they are not vulnerable to data loss in case of a power outage. How is the situation with Mac Minis which don't have a battery? Are there multiple redundant power input the Mac Mini can switch in-between without tuning of?

      2 replies →

  • EugeneOZ  4 years ago

    This means the world would be full of complaints from macOS users, but for some reason, we only know about this detail because of that “shocking” Twitter thread.

    • marcan_42  4 years ago 

          #!/usr/bin/python
    import os, sys, time, datetime

    t = datetime.datetime.now().isoformat()

    print(t)
    for i in range(5):
        time.sleep(1)
        print(4 - i)

    fd = os.open(sys.argv[1], os.O_RDWR|os.O_CREAT)
    os.lseek(fd, 0, 0)
    os.write(fd, b"test: " + t.encode("ascii") + b"\n");
    os.fsync(fd)
    print("done!")
    time.sleep(100)

      Run that on a Mac Mini. Do it a couple times. Remember the timestamp of the last one. Let it count down, then pull the plug within a few seconds after "done!" shows up. Boot up again. The file contents will have reverted to a prior point in time.

      This isn't some hypothetical thing, this is a trivial test you can do. fsync() on macOS does not guarantee data is on stable storage. And this is actually well documented.

      Then if you want to see the performance problem, make it a loop instead and use `fcntl.fcntl(fd, fcntl.F_FULLFSYNC, 1)`. You'll get 40-odd IOPS, but at least your data won't disappear after power loss.

      8 replies →

    • layer8  4 years ago

      Looking at the Apple support forums, the world is full of complaints about all sorts of malfunctions (which may or may not be related).

      1 reply →