← Back to context

Comment by publlus_enigma

10 hours ago

There are four times in my life I have suffered regrettable data loss incidents.

The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover everything.

The second was storing data in OneDrive - a change to their terms surrounding "lifetime" unlikely noted storage, combined with a client that was unusably slow to download and a deadline for data retrieval meant that I lost most of my files.

The third was SD card failure in digital camera on holiday, the controller chip died catastrophically, leaving the card completely unrecognised. It was a brand new Sony 128GB card, manufactured by Toshiba, and it seemed to be a common issue. I now shoot to two cards simultaneously.

And the fourth time was ... Performing a backup. An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup. I've been weary of using rewritable media for some time now as a consequence, but I think backups themselves are high risk activities.

53 comments

publlus_enigma

Reply
Dylan16807  4 hours ago

I would say you didn't get quite the right lesson from the fourth one. Rewritable devices are fine, but have (at least) one backup completely unplugged while you update another backup.

lolakutty  5 hours ago

>I did have backuos, but only only floppy and they didn't cover everything.

This made me remember the time during the 90s when I lost all my BASIC programs due to a hard disk crash, and seeing me sitting sad, my mother handed me a bunch of floppies and asked me to see if they contain something useful.

And turned out that it had all the programs! I took the backups and forgot about that. Apparently that is one of the hard things with backups. You need to track them...

cvhc  10 hours ago

> a change to their terms surrounding "lifetime" unlikely noted storage

I had a similar experience with a Chinese cloud storage provider. They didn't even give a convenient way to export the data. And the client throttled download to like 100KB/s. I luckily was able to parallel download by running the client on several VMs...

> An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup.

Sounds like the backup workflow is fundamentally flawed... But I also have the fear that the backup program with root privileges can go off the rails... So I keep my backup job as simple as just running restic with some systemd restrictions.

Gigachad  9 hours ago

>I've been weary of using rewritable media for some time now as a consequence

I briefly considered using bluray disks as a backup for my photos and other critical docs. But getting a decent bluray burner seems not so easy these days with most production winding down. Next best thing looks like the "object lock" feature on object store services that prevents deleting objects for a certain time.

  • Terr_  8 hours ago

    I pretty much instantly wrote off physical media because I know I won't reliably move it to a separate location. If I'm going to have backups, I want ones that are not within fire/flood range of my computer.

    In terms of preventing "oops" moments, I'm mainly relying on software (restic) for that, where I trust that (A) backups always append data rather than replacing and (B) it's logic works for marking which data to purge based on rules is accurate. [0]

    [0] https://restic.readthedocs.io/en/stable/060_forget.html#remo...

    • Gigachad  3 hours ago

      You want to make sure the append only is enforced by the hosting provider rather than duplicity. Your backups should be resistant to having an attacker on the system who can run arbitrary commands with duplicity.

  • cyberax  8 hours ago

    I bought a tape library for my backups, but that's probably an overkill.

    On the other hand, LTO-5 drives are now pretty affordable. And each tape cartridge holds around 2Tb of data for about $20.

  • justsomehnguy  7 hours ago

    > Next best thing looks like the "object lock" feature on object store services that prevents deleting objects for a certain time.

    Lol, no. "As of today we are closed. Goodbye."

    Especially considering what giving even a two weeks now considered "generous".

    • Gigachad  3 hours ago

      This is for backups. The threat here is that random ware uses your backup script to delete/encrypt the backups too. With object lock the api key the script has can only add data.

      The probability that the object store goes out of business at the exact time your own copy dies is insignificant.

      2 replies →

dsego  6 hours ago

For digital media we really need a good stable archival storage solution. Nothing on the market right now for ordinary people.

baxtr  5 hours ago

Since we’re talking about it: anyone backing up their iCloud photos? If so, how?

  • wallst07  28 minutes ago

    Amazing question with nobody really answering your question 100%.

    The answer is : it's hard. If you only have a iphone and more photos on your icloud than can fit on your phone, you're kinda screwed if you don't know what you're doing.

    AFAIK Apple makes it very difficult to get all your icloud photos bulk downloaded, original quality off icloud.

    Google is no better.

    It's much easier to start backing up from the beginning but once you're stuck on icloud it's hard.

  • reddalo  3 hours ago

    It's not iCloud Photos, in my case it's Google Photos. But I keep a copy of all my photos in a self-hosted instance of Immich [1], which I then back up to a Hetzner Storage Box using Backrest [2].

    So I end up with 3 copies of all my photos, on 3 different providers (I mean... 2 different providers plus my homelab disk).

    [1] https://immich.app/ [2] https://github.com/garethgeorge/backrest

  • unanimous  3 hours ago

    When I was using Ubuntu, I would plug my phone into my laptop and run this small script: https://gist.github.com/wheelercj/8cce112fc8ee55fd4858847a52...

    It's mainly just rsync. When the phone was plugged in, the photos were accessible at a path like "/run/user/1000/gvfs/gphoto2:host=Apple_Inc._iPhone_abc123/".

    Now I use Time Machine, but I still need to check whether it includes iCloud photos.

  • mmazzarolo  5 hours ago

    I have a weird setup lol. I use a shared photos library with my partner (so that we have all family photos together) and have a MacBook Air that has Photos being download in raw quality and backups then via Time Machine on a Synology NAS. Plus, Synology photo sync on my iPhone (backing up on my NAS separately from Time Machine) AND Google Photos on my partner’s (syncing on Google Photos). Yes it’s horrible, just never had time to clean I up properly but I prefer to have them in multiple cloud locations and locally for now than optimise for having a clean setup.

  • undebuggable  4 hours ago

    From iPhone into local machine through LocalSend or AirDrop, then your usual favorite backup pipeline.

Forgeties79  10 hours ago

>I now shoot 2 cards simultaneously

A very lovely change over the last 8 years or so lol. I came up in film during the DSLR revolution. 5D2’s/7D’s/Rebels (i series) years.

8bit 420 nasty aliasing recording on single cards and praying baby. Magic lantern booted on those same SD’s!