← Back to context

Comment by guessmyname

7 years ago

This reminds me of the long conversations that I used to have with family members and friends several years ago. With their continuous requests to create my own Facebook profile so I can keep in contact with them and with their activities as well as to share my whereabouts. I always used the same argument to reject these suggestions — "I don't want Facebook to have too much data about me, more than the data that you already provided".

I got used to the looks of disbelief, thinking that I was some sort of hermit, an antisocial.

I also got tired of answering the frequent "Why don't you have Facebook?" questions.

I remember the last time I had this conversation with someone, last year (2017) around August. I found a new love partner, and after the long intimate talks on the phone, they requested the usual "intimate pictures", not necessarily sexual but certainly sexy. While I have no tabus with regards to my sexuality, having an understanding of how the Internet works, I have always refused to send that type of images/videos/audios, and I always tried to be patient with the other person to explain my constant denials. Unfortunately, expecting a non-tech-savvy person to understand how data moves around the Internet is most of the time based on hope, and even if they understand, they ultimately don't care because the result doesn't change: you don't get to share something with them and that affects personal interactions.

I am sure that the deletion of media files in services like Facebook has never meant to be absolute. Many of my colleagues believe the same thing that I believe: Facebook and other services do not actually delete data, they just mark it as "deleted" and purge it only if they need the space. The same way a hard drive works, you don't really delete a picture when you hit the "delete" key, nor even if you clear the "trash" folder, the data is still there, where it was, it just loses the links to the metadata.

It is sad how this information becomes news only when bad things happen.

> I am sure that the deletion of media files in services like Facebook has never meant to be absolute. Many of my colleagues believe the same thing that I believe: Facebook and other services do not actually delete data, they just mark it as "deleted" and purge it only if they need the space.

No reason to believe. You can read about the storage architecture used to store photos from a post in 2009 here: https://code.facebook.com/posts/685565858139515/needle-in-a-.... Obviously that might and probably has changed since, but at least at some point that was exactly true.

Quote:

"The delete operation is simple – it marks the needle in the haystack store as deleted by setting a “deleted” bit in the flags field of the needle. However, the associated index record is not modified in any way so an application could end up referencing a deleted needle. A read operation for such a needle will see the “deleted” flag and fail the operation with an appropriate error. The space of a deleted needle is not reclaimed in any way. The only way to reclaim space from deleted needles is to compact the haystack (see below)."

  • Allow me to ask the obvious question.

    Who doesn't do the something like this?

    Not to alleviate facebook of blame, but who's to say data on almost every other social media service isn't also just flagged for deletion?

    • We don't soft delete payloads at Raygun (https://raygun.com), for the very fact that typically if one of our customers wants to delete something it's because they might have sent something they don't want a third party to have. We have filters and other PII filtering tools etc, but it every now and then something might be sent by mistake.

      Having said that, you'd be amazed how often folks ask for things to be undeleted (despite a big warning dialog).

      Clearly developers pervasively believe soft deletes are occurring everywhere.

      12 replies →

    • > Not to alleviate facebook of blame, but who's to say data on almost every other social media service isn't also just flagged for deletion?

      The word "delete" has a pretty clear definition to most users. Facebook is one of the most used pieces of software in the world. If FB is allowed to lie to its users, it would indeed give a pass to just about every social media service out there.

      The reason Facebook is special, and deserves special scrutiny, is because of its power. If FB establishes a bad behavior, it will become the norm.

      2 replies →

    • Does it make it ok for Facebook to do it just because similar other companies do it? I say no, all of them should delete something I say to delete. And "everyone does it" is makes it a bigger problem, not a smaller one.

      11 replies →

    • I recall the distinction being made very clear on LiveJournal between "deleted" content vs "purged". I would be very surprised if they were not being forthright about this. Of course this was 10 years ago, before the Russian ownership. So I do have reason to believe that not all companies act in deceitful ways when it comes to retention of user data.

    • Yeah i don't see much difference between this and hitting delete on a file in a local file system. The data itself still sits there until the sectors gets reclaimed, but there is no longer a file name or directory entry associated with them.

  • The database we use (Vertica) works this way. Nothing is deleted. Instead it is flagged as deleted. A background task may purge old data (older than x). Historical queries show the database state as it was days or weeks ago. If the background task is broken (bug?) then the data stays indefinitelly on disk.

  • So, like a filesystem more or less

    I might just "help" them by uploading more data I guess

Unfortunately there will be no prizes for having been right all along.

Even now, as facebook is burning, statements of how one has quit or will be quitting facebook get swept into the pile of incendiary indignation, with encouragement from all sides.

But never having used facebook, even at significant personal effort as you indicate, one is relegated from "elitist" before to "smug" now.

One day in the future a recruiter will ask why there's nothing about you on the Internet, and you will proudly be able to say: "Because I know the Internet and its dynamics that well" and they will hire you, in awe of your analytical foresight.

That's the dream anyway, because you're more likely to be reported for being suspicious. After facebook there will be another facebook, and another, and people will flock to them just the same, and you get to experience being an antisocial hermit all over again.

Now I made myself sad. "Social Media: even more depressing when you're not on them!"

  • > Unfortunately there will be no prizes for having been right all along.

    Except not having your racy pictures in Facebook's media archive.

  • What about the option where people return to messaging applications for private matters and keep a Tweetbookdin for public persona ?

  • Facebook is burning. Lol. They suffer a minor setback and they're "burning". Right now there is not much of alternative to facebook, it will be just fine.

"I am sure that the deletion of media files in services like Facebook has never meant to be absolute." This is very common, I'm sure. There should be a way to request or a right to request permanent deletion, by law, of one's data on site like Facebook. That said, once something is on the internet, anyone can and will archive it (see https://www.reddit.com/r/DataHoarder/). Closing an account, however, should imply permanent deletion. Companies are instead able to operate in a gray area through terms of service agreements that knowingly play on the ignorance of the end user. This common and widespread behavior is a detriment to the user and (arguably) society at-large.

  • Obviously I'm not privy to the details of this particular requirement, but I'm fairly certain that very few, if any, of our videos actually go away when we delete accounts. (Or even when we delete the videos themselves.) I think this because I've seen images from SMS texts, instagrams, snapchats and things of that nature used in court cases. So law enforcement must have access to that stuff somehow? But, again, I'm not privy to the technical or legal mechanisms they use to make that happen. All that said, I have seen images from services like these in court cases. And defendants have CLAIMED that they had deleted them. (For whatever value of "deleted" exists on the given service.)

    So I'm wondering if the services actually have some sort of archiving requirement for law enforcement purposes? Maybe for a certain number of years, they have to save your data or something like that?

    If there's anyone who would be familiar with the legal obligations of these services vis-a-vis data archiving I'd be really interested in hearing more about what we should reasonably expect from these services in terms of deletion etc?

    • > So I'm wondering if the services actually have some sort of archiving requirement for law enforcement purposes? Maybe for a certain number of years, they have to save your data or something like that?

      Apart from a handful of specific cases like financial data, the US has no general data-retention laws. You can delete stuff aggressively as long as it's based on a consistent archival policy, not one-off deletions where you risk looking like you chose a particular thing to delete to hide evidence.

      You can tell this is possible in practice by looking at how common it is to have aggressive permanent-deletion policies in corporate email, at least outside of tech. A number of big US companies automatically delete read emails in employees' inboxes after N days (with N ranging from 7 (!) to 365), unless the employee specifically takes action to refile the email into a project folder with a different per-project retention policy. The goal of those policies is to reduce companies' exposure to fishing expeditions in future lawsuits by just keeping less email around. To make that effective, the policies really do delete the emails, including from any backup systems.

      Given that they have figured out how to perma-delete their own old email, I believe companies could really delete user-deleted content, perhaps after some specified period of time, if they wanted to. But unlike with their own internal emails, they don't have the same incentives to be aggressive about purging that stuff from their servers. If anything, they have the opposite incentive, to keep as much user data around indefinitely as possible.

  • GDPR is intended to at least force service providers to give folks the right to be forgotten which compels providers to delete data. While it's own Europe, it's difficult to comply without just making general decision about honoring these requests.

    • Actually, GDPR only requires that any links from the data to the user should be destroyed, so that you can no longer figure out who created the data. This means that a lot of data will be left. And realistically I think that a lot of it will remain identifiable, just like anonymized data can be traced back to real users pretty easily if you have enough data points.

      11 replies →

    • The right to request deletion of data is already mandated in current laws, GDPR doesn't change much in that regard.

      The hard part is actually enforcing it, and assessing compliance.

> Facebook and other services do not actually delete data, they just mark it as "deleted" and purge it only if they need the space.

You may be correct, but that doesn't explain why Facebook decided to include so-called deleted files in a download of user data. Clearly these deleted files are still a part of Facebook user profiles and accessible to company data mining software. Facebook has exposed their own duplicity.

  • Maybe the Facebook development processes and tracking of tech debt is just shit. First person: "I'll just flag the content and then it won't show on their timeline!" Second person: "I'll just select all the records that belongs to this account when packaging a backup. All the deleted content should be gone!"

    But I wouldn't discount your hypothesis.

    • When storage is cheap, it's rational to develop the delete flag first and think about cleanup later, which means never. The download content thing seems like a low priority project and the poor intern who probably did it didn't want to figure out how each store keeps the delete flag. At least it's honest. Would you be surprised a dd of your sd card showed your deleted photos?

      12 replies →

  • >Facebook has exposed their own duplicity.

    Or possibly they just screwed up. Perhaps the "soft delete" was originally intended to allow "undelete" by the user with delayed purge, and/or single-instance storage with reference counting that they never quite got around to finishing.

  • > but that doesn't explain why Facebook decided to include so-called deleted files in a download of user data.

    This happened because the person tasked with writing the code to build the archive forgot to include the filter for "deleted" records somewhere in the code.

    I.e., they forgot the "where is_deleted = false" part below on one or more DB query requests like this:

    select * from table where is_deleted = false;

    This is the biggest problem with the "soft delete flag in database" method of deletion. Every single query writer, everywhere, forever, must always remember to include the "is_deleted" filter in their queries. And when they don't, what was deleted reappears as if it had never been deleted at all.

Facebook is powerful and insular. Taking it down requires extraordinary organisation. Outrage is helpful in that respect.

Agreement is better than disagreement. Would I prefer we had agreement earlier? Yes. Is agreement today better than agreement tomorrow? Absolutely.

Now that we have a constituency, the important thing is to mobilise. The past is in the past. Our job, in the present, is to protect the future.

  • Call for a facebook user strike on May 1st.

    #May1FBstrike

    https://medium.com/@oddbert2000/call-for-a-facebook-users-st...

    • Excellent - they'll be able to offer advertised another demographic category "People who are vocally privacy conscious, but who aren't prepared to do anything about preserving it if it means they don't get to play Farmville."

      Antivirus vendors and shitty vpn services will be all over that.

    • Why not just every day? Only use it as much as absolutely necessary (to communicate with people you wouldn't be able to reach otherwise) and use competitors instead. Even using FB owned companies (e.g. Whatsapp) would help. While FB still gets some data, they don't get contents (unlike FB messenger all chats are end-to-end encrypted) and most importantly no ad revenues. And lower revenues is what would truly change Facebook's policies.

  • You think people could organize better than against Trump ? seems unlikely

    • > better than against Trump?

      It is more effective to organise for a cause than against a politician. Presidents are intentionally difficult to remove. The bar for promoting action against Facebook is lower than for prompting action against the President.

> It is sad how this information becomes news only when bad things happen.

What bad things? I feel that's the part missing from the argument. People have yet to see or hear what are the negative consequences of all that data being kept or even leaked or re-sold.

The only one they've started to know about is the potential impact on elections, which is pretty hypothetical and weak to most people I feel. Or maybe identity theft, but that's more related to the Equifax leak.

I think its important to rationalise on what are the real consequences of our data no longer being private. Is it really dangerous? What's the worse that could happen? What are the chances of it happening, etc.

> I have always refused to send that type of images/videos/audios

Isn't it still trivial to self-host stuff?

Just send a link to picture (or document or whatever confidential information you want to share) to a password-protected resource on your own server (or even a laptop or desktop machine, if you have globally routeable IP address there). Facebook automation is not that smart to grab the password from the very same conversation, and even if they do - I'm sure they won't do it, knowing you'll catch them in access logs and press charges for unauthorized access.

I doubt many would object and insist on sending via a very specific medium (i.e. strictly require pics in a FB Messenger). Some, of course, may find this inconvenient.

  • "trivial" and "your own server" together? :) Maybe for some code monkey, but not for my mom. :(

    I really do wish self-hosting were more trivial, it would be a better world.

  • Where I live Internet providers deliberately make self hosting anything extremely hard.

    Then they charge often 5x or more their normal price to let you host things, but add lots of exceptions, for example all providers put in contract they can immediately cancel your subscription of they detect you hosting anything irc related, doesn't matter of it is a irc server or a irc bot or a server for a open source irc client...

I think a great way to explain privacy limitations to a non-tech-savvy person is to walk them through using GPG.

Once someone understands public and private keys, and webs of trust, there really isn't much left to learn. For someone who understands keypairs, the limitations of Facebook/Twitter/etc., DRM, etc. are obvious.

It seems most of us are afraid our non-tech-savvy friends and family won't be able to wrap their heads around security, but not understanding it has gotten us into a pretty bad situation. We should really stress the importance of learning about it.

  • > Once someone understands public and private keys, and webs of trust

    Nobody in the general public wants this.

    • Okay, don't assume people won't be interested in interesting things. Who is this general public, anyway? It's not an homogeneous group; it's made up of physicians, mechanics, teachers, lovers, Doomsday preppers, engineers, preachers, and all kinds of people who have special interests. The thing I see is that if you show them how it matters to them in their special role, rather than to them as members of this general public, they may well take an interest. Some of them may become very deeply interested indeed, if they needed such a thing but didn't know about it until you showed them!

      2 replies →

    • An awful lot of people in the general public do.

      Especially if their tech-savvy friends are confident they can learn about it - because it really isn't that complex - and if they understand that keypairs and trust are the basis for literally all digital security.

  • > great way to explain privacy limitations to a non-tech-savvy person is to walk them through using GPG.

    Have you ever actually successfully done this? More than once? And they continue to use it?

    It's a usability nightmare. https://moxie.org/blog/gpg-and-me/

    • OK. Maybe not GPG specifically.

      But keypairs. Everyone should understand keypairs. They are the basis for all of digital security, and they are really not that difficult.

      1 reply →

  • Lol. Sure, and the best way to teach my grandma about computers is to install arch linux on her pc.

>I got used to the looks of disbelief, thinking that I was some sort of hermit, an antisocial.

I know that look.

>I also got tired of answering the frequent "Why don't you have Facebook?" questions.

I solved it by stating flatly "For the same reasons I don't have Twitter.", somehow marking the final period, people still believes I'm a kind of weirdo, but they don't go on asking ...

>Unfortunately, expecting a non-tech-savvy person to understand how data moves around the Internet

Explain how data can be unreadable while it moves. Teach them to use secure communication options. You don't need to be an electric engineer to use a TV remote control.

  • But no one has made a TV Remote control version of "encrypted Facebook" or even "encrypted eMail".

    And heck, there are people who can't use tv remote controls.

    The only thing that I'd consider "easy" is encrypted chat (signal). The "issue" there is market fragmentation (arguably a good thing).

I always tell people to treat Facebook as if every person you ever meet will be able to see it. It's more or less my public persona. Twitter is more anonymous.

  • Unless it's encrypted and ephemeral, treat every bit sent out to the internet, a public network, as your public persona.

    • Yep, another way of looking at it is if it leaves your device, assume the information is eventually open to the public.

    • Amen. Once you upload it, you should just assume it’s out there forever. It’s probably worth assuming that virtually all anonymity can be pierced, if not now then within a decade or two.

      With a few exceptions, anonymity online is ephemeral at best, subject to the motivation of the person/org trying to deanonymize you.

  • > Twitter is more anonymous.

    How did you arrive at that conclusion? I assume Twitter retains everything as well (even "deleted" tweets) and it's all associated with an email address. Or did you mean it in the sense that far fewer people have a Twitter account?

    • I used to believe Twitter was better. But once you're above a certain number of active (i.e. publicly retweeting) followers there's a pretty high chance that your tweets will end up in the feed that is used to generate the twitter stream archives:

      https://archive.org/details/twitterstream?sort=-publicdate

      These are tar files that contain bz2 compressed newline separated twitter events as json. These include deletion events as well, so you can for instance easily estimate the time an auto-deleter is set to.

      Yes, they're huge archives, but you could still probably process a year of these for particular targets for under $10 on EC2.

      Whilst I'm impressed with archive team's efforts, I would be surprised if there aren't some commercial twitter stream consumers that absolutely dwarf this.

      Treat everything you put on twitter as public forever and you won't go too far wrong.

      2 replies →

    • Twitter is more about finding your own social graph of people you find interesting than friends/family/coworkers like Facebook is primarily about. I could have a completely anonymous persona on Twitter and get all the same content. I could use a fake name on Facebook but it wouldn't make as much sense, and I could be reverse engineered with some accuracy from just my social graph. Your family is going to tag you as family, etc. The other non family and friends content on Facebook is more watered down than on Twitter and Facebook wouldn't be worth using for that alone.

  • How is Twitter more anonymous? In the UK people have been locked up for tweets.

    Twitter probably have less data on you, but I doubt it can't be linked direct to you by a TLA, say.

>I remember the last time I had this conversation with someone, last year (2017) around August. I found a new love partner, and after the long intimate talks on the phone, they requested the usual "intimate pictures", not necessarily sexual but certainly sexy.

Why the fuck are these a thing? Couples don't meet in real life much anymore? And how "usual" are they?

  • Anyone have stats on how widespread this is? My spouse and I avoid being in front of cameras naked even when we're pretty sure the camera isn't enabled. Not that anyone else would really want to see us nude, but why take a chance on accidentally recording material that could be embarrassing?

> expecting a non-tech-savvy person to understand how data moves around the Internet

Then we - the people that do have the necessary technical knowledge - have a duty to teach them what they need to know. This isn't necessarily "how data moves on the internet". Yes, this can be difficult and tedious, but understanding the risk profile for data/networks is increasingly important as networks become involved in everything.

> they ultimately don't care

Again, it's our duty to teach them why they need to care. This probably shouldn't involve a lecture on networking or data analysis, but instead tailoring an explanation to their personal situation and knowledge.

  • I don't think it's because they don't understand or because they don't care, it's just overwhelming. Think about it, to have any basic grasp of understanding regarding the security infrastructure of the internet you need to have a basic understanding of network connections, how HTTPS works, how files are stored on your computer, how files are sent across computers, how your average database works etc...

    Think about the last time you've tried tinkering with something you're a noob at. Maybe it's deciding that you would try fixing your car engine yourself even though you never were a mechanic. Maybe you decided to make a complicated cake and halfway through you realize that you overestimated your pastry skills. Try to remember the feeling of helplessness you felt at that moment, the "I have no idea what I'm doing and I wish I never had started that in the first place". In my experience that's how 90% of people feel like when trying to do something technical with a computer.

    A few weeks ago a colleague from HR asked me if I could make a backup of a computer because it contained some critical stuff and she wanted to be able to restore it later if necessary. I say okay, boot up a debian live USB stick I had lying around and start dd'ing the drive to external storage. When I told her the copy was in progress she told me "but I didn't give you the password?". She was amazed when I told her that I didn't need the windows session password to access the data on the disc. I swear I'm not making it up when I say that she asked me if I was a "hacker".

    That made me realize that there are probably many people out there who think their files are safe as long as their Windows password isn't compromised even if the disc is not encrypted. After all, they can't access the files, so surely nobody else can? If Facebook says my photo is deleted, then surely it must be? Why wouldn't it be?

    I don't think it's fair to blame these people, we've designed so many strange patterns over the past decades in software that it's difficult to keep track. Maybe having "delete" not actually delete should be considered a dark pattern. Maybe it should even be illegal.

    • "That made me realize that there are probably many people out there who think their files are safe as long as their Windows password isn't compromised even if the disc is not encrypted."

      Of course they assume it. Partly also because windows tells you, if you loose your password, you can no longer access your account, which is bs and they know it and tell you only for "felt Security".

      And encryption ... What is that?

      2 replies →

  • And how would we do that? Every time I've tried to explain privacy issues to non tech individuals at best they consider me paranoid and at worse a fucking sociopath who doesn't have a FB profile because I can't correlate with other people. I can't carry this burden and I doubt many can.

    There have been horror stories over the years about identity theft, even before the emergence of social media. Has this stopped anyone outside our community from posting details about their lives online? I hardly think this whole situation with FB will change anything in the end.

    I don't feel I have any obligation/duty towards anyone. If they want my opinion or ask me about an issue I'll gladly inform them. But I won't start a crusade for a better informed society. Internet was supposed to do that and we ended up with videos of cats and wannabe celebrities posing seminude pics on Instagram. Fuck that shit.

    • Your view is well represented on the Internet, and is perhaps most aptly exemplified by the early jargon word “luser”, and the BOFH phenomenon. I have never, I think, really been prone to such thinking. I have never had a problem talking to ordinary people or users, or felt the immense frustration which many people have vividly described. (Note: I am a sysadmin with approximately 20 years of professional experience, and have always had a user-facing role as at least a part of my job.)

      It reminds me where in Zen Buddhism, there are those who become enlightened and go off to do their own thing, and those who become enlightened and stay in the world with the rest of the ordinary unenlightened people. In the words of Alan Watts:

      The understanding of Zen, the understanding of awakening, the understanding of– Well, we’ll call it mystical experiences, one of the most dangerous things in the world. And for a person who cannot contain it, it’s like putting a million volts through your electric shaver. You blow your mind and it stays blown. Now, if you go off in that way, that is what would be called in Buddhism a pratyeka- buddha—“private buddha”. He is one who goes off into the transcendental world and is never seen again. And he’s made a mistake from the standpoint of Buddhism, because from the standpoint of Buddhism, there is no fundamental difference between the transcendental world and this everyday world. The bodhisattva, you see, who doesn’t go off into a nirvana and stay there forever and ever, but comes back and lives ordinary everyday life to help other beings to see through it, too, he doesn’t come back because he feels he has some solemn duty to help mankind and all that kind of pious cant. He comes back because he sees the two worlds are the same. He sees all other beings as buddhas. He sees them, to use a phrase of G.K. Chesterton’s, “but now a great thing in the street, seems any human nod, where move in strange democracies a million masks of god.”

      — Alan Watts, Lecture on Zen

  • > ... instead tailoring an explanation to their personal situation and knowledge.

    I’ve used this with success several times. Though you generally have to know the person well enough to know their “secrets”.

> I am sure that the deletion of media files in services like Facebook has never meant to be absolute. Many of my colleagues believe the same thing that I believe: Facebook and other services do not actually delete data, they just mark it as "deleted" and purge it only if they need the space.

This is a dumb conspiracy theory. Facebook has made plenty of public statements that say otherwise, and there's a whole team that works on the system that ensures every trace is erased from disks, logs, cold storage and backups when deleting content.

  • Looking online briefly for definitions of "delete":

    "remove or obliterate (written or printed matter), especially by drawing a line through it or marking it with a delete sign."

    "synonyms: remove, cut out, take out, edit out, expunge, excise, eradicate, cancel"

    All of these seem clearly "absolute" to me. "Delete" means it's gone.

    I think Facebook has its own special linguistic distortion field. It requires no "dumb conspiracy theory" to realize that Facebook cannot be trusted.

    • Deletion by flag is very common in IT and presumably has been since the first undelete program was created. It's not a Facebook thing.

      Some mail programs for a long time have had a soft-delete that requires an expunging process to create compete removal.

      In an IT setting you can delete a blob from a db, but it might still be on disk, and it will still be in caches, on user machines, and in backups/archives.

      1 reply →

  • I'm not inclined to believe PR statements like these when there's no way to verify them.

    Can you support your assertion? The infrequent cases where someone manages to extract or recover supposedly deleted data cast a lot of doubt on your claims.

    In any case, even if it's not Facebook specifically, it seems overwhelmingly likely that the majority of companies do not actually delete your data.

    • FB is/was regularly audited by Irish DPC, I think one of the topics was user data deletion. I think that the results were public.

  • To be fair though, the article that this comment thread is attached to offers some seemingly direct evidence to support one aspect of this 'dumb' 'conspiracy' 'theory'.

  • Did you read the OP? How can you say that this is a dumb conspiracy theory?

  • First lesson in DB class: do NOT delete. Just flag.

    I can give you plenty of statements about how I'm Santa Claus though.