Comment by EugeneOZ

4 years ago

This means the world would be full of complaints from macOS users, but for some reason, we only know about this detail because of that “shocking” Twitter thread.

11 comments

EugeneOZ

Reply
marcan_42  4 years ago 

    #!/usr/bin/python
    import os, sys, time, datetime

    t = datetime.datetime.now().isoformat()

    print(t)
    for i in range(5):
        time.sleep(1)
        print(4 - i)

    fd = os.open(sys.argv[1], os.O_RDWR|os.O_CREAT)
    os.lseek(fd, 0, 0)
    os.write(fd, b"test: " + t.encode("ascii") + b"\n");
    os.fsync(fd)
    print("done!")
    time.sleep(100)

Run that on a Mac Mini. Do it a couple times. Remember the timestamp of the last one. Let it count down, then pull the plug within a few seconds after "done!" shows up. Boot up again. The file contents will have reverted to a prior point in time.

This isn't some hypothetical thing, this is a trivial test you can do. fsync() on macOS does not guarantee data is on stable storage. And this is actually well documented.

Then if you want to see the performance problem, make it a loop instead and use `fcntl.fcntl(fd, fcntl.F_FULLFSYNC, 1)`. You'll get 40-odd IOPS, but at least your data won't disappear after power loss.

  • EugeneOZ  4 years ago

    Does it somehow prove that people are losing files when transferring them over the network? People would definitely notice it.

    • marcan_42  4 years ago

      You're very persistent, aren't you?

        marcan@raider:~/tmp -$ echo "very important data" > file.txt
  marcan@raider:~/tmp -$ rsync --remove-source-files file.txt macmini.lan:

      Yanked power to macmini.lan after the rsync completed, then turned it on again

        marcan@raider:~/tmp -$ ls file.txt
  ls: cannot access 'file.txt': No such file or directory
  marcan@raider:~/tmp 2$ ssh macmini.lan
  Last login: Thu Feb 17 23:45:26 2022 from 192.168.3.10
  marcan@Mini-M1-2020 ~ % ls file.txt
  ls: file.txt: No such file or directory

      Data's gone.

      This is real, please stop pretending it isn't.

      6 replies →

layer8  4 years ago

Looking at the Apple support forums, the world is full of complaints about all sorts of malfunctions (which may or may not be related).