← Back to context

Comment by preg_match

2 days ago

I think the bigger question is whether this is even worth solving. If people really want to give someone root access and do it willingly, maybe we just say this is out of our purview and allow it. And then, slowly, people will learn. They will be hard lessons but evidently baby proofing the entire world is not really working, so we might just abandon ship on that idea.

> And then, slowly, people will learn

I hope you're right, but I'm not sure they will. The less locked-down WinXP and early Android era was ... really really bad in terms of nontechnical people getting hacked/scammed. And bad actors are also innovating, so the target of what people have to learn is moving: now we have synthetic AI voice scams, easy-to-create full clones of popular websites that harvest credentials, an explosion of 0days that let people run RATs that hassle elders for their MFA codes, and so on.

Like, I'm 100% with you that baby-proofing the world is an unattainable, patronizing, and negative-externality-laden goal. We shouldn't do that. But we shouldn't go full libertarian "you're on your own; toughen up" either; I think we have a lot of data that indicates that the harms of that approach are both high-magnitude and high-volume.

  • On the sliding scale here where the left is "full anarchy" and the right is "goo goo ga ga baby proof the world", Apple is about 95% to the left. Here's how that looks:

    Anarchy |--------O-| Goo Goo Ga Ga

    And people are arguing we should be moving further right. It's ridiculous, we all need to be candid and recognize this will not work

    • Fortunately, it's not a linear scale. All sorts of technical and political options exist which don't fall cleanly into the anarchy or baby-proofing spectrum. Random incomplete examples, in no particular order and off the top of my head:

      GGP's idea of making it more commonplace to have a "Johnny" helping people with their tech needs to prevent accidents could work socially, if there are ways for communities to create more people willing to do that.

      Worldwide legal penalties for spamming/scamming could grow more teeth, increasing the likelihood of bad outcomes for people that make malware or questionable apps.

      Software distribution systems could standardize on better systems of provenance and ownership handoff to further technically harden against "good extension sold out to an evil maintainer" or "github credential leak let a bad guy publish an artifact"-type attacks.

      Cooldown periods for users trying to grant questionable access patterns could be imposed, though that might feel too baby-proof for some.

      On-device permission boundaries could be modeled in an "XOR" way: apps distributed from Apple's walled garden could be disallowed from approving data sharing with apps users install from other repositories. That's Apple's prerogative (they own the distribution and vouch for at least some of the quality of the app store apps), but doesn't prevent users from installing parallel ecosystems if they want.

      Something that's very paternalistic, but doesn't involve baby-proofing what's possible, is the idea of credentialing users. You need a driver's license to operate a car. In the US, you need a (much easier to get) food handling license to commercially process food. The latter's a short briefing and test of comparable effort to those mandatory corporate anti-phishing trainings that already-technical people hate. Perhaps some users could benefit from that as a prerequisite to installing non-trusted software.

      For vetted walled gardens like the App Store, further improving the granularity of and required justifications for permission requests as providers have been doing might help. "This poker app wants to access all of your saved contacts and photos" becomes "this poker app wants you to select a single profile photo and up to 5 contacts a day to add as opponents, after which access is revoked" or whatnot. This only works if App Store reviewers get serious about rejecting apps for overbroad permissions requests and explain their rationale to app developers, which would require Apple and friends to spend a lot of money to enable. Fortunately, they have insane margins. Less fortunately, their shareholders wouldn't go for this unless forced by regulation or similar.

      Anything that helps with threat attribution. If Grandma can install an app from a random URL, and then gets hacked 6 months later, it'd be great if something got in her face that loudly indicated that the decision to install the untrusted app was the root cause of her compromise and some "do you want to disable the ability to do this in the future/require a phonecall to $provider to turn it back on?"-type hint.

      For apps that spend money, further integrating pattern-aware anti-fraud and spend caps with payment APIs so that e.g. a microtransaction app that got hacked can't suddenly spend $100 where the user typically spent $5/month. Banks are already starting to get proactive/argumentative about unexpected transaction patterns a la "sir, are you sure you want to send $5000 in your first overseas money wire? Can you tell us more about that transaction? Are you aware of this common fraud?"

      ...and so on.

      1 reply →

Tell me that when it’s your relative that is out thousands of dollars because they were scammed.

  • I'm not going to argue we should close all banks or whatever ridiculous argument people come up with just because someone lost money.

    The only way to 100% prevent scams is to remove the potential entirely. There are scams, infinite actually, right now, on iPhones. Do you support that? Surely not, so we should lock iPhones down more no? Ideally, we ban smartphones altogether, to prevent scams. Oh you don't support that? Well come back when one of your relatives loses 1 million dollars.

    Look, we can take steps to prevent scams, and we do. What we should NOT do is go so extreme that people cannot even use their devices the way they want that they paid for. This is anti-consumer. If I want to install some software, I should be able to. It is not Apple's responsibility to baby me, baby you, and baby the entire world and say "no no you can only use our approved software!"

    Especially when some of that approved software is... wait for it... malware! Yes, there is quite a lot of malware on the Apple App Store and the Play Store. But I can't install open-source software I've audited myself on my iPhone? How strange.

    It appears to me this has nothing to do with scams, and everything to do with control, censorship, and profiteering.

    • The comparison to banks is actually pretty interesting.

      Unlike app developers, banks are by default directly liable for fraud that happens on their system. In many cases, even when someone gets their bank to send money to a scammer, they can still get that money back. Since the bank doesn't want to risk regulatory reprisal, they have a lower threshold for spending money to make customers whole. Exceptions and subtleties abound here, but the underlying incentive structure is very different: the broker of the sensitive resource (money, for banks) is often held responsible for mis-use of that resource. How would we do that for e.g. apps that store your password in plaintext and then get hacked?

      Relatedly, banks have thus started adopting the practice of blocking and calling/talking to customers about suspicious transactions. You can still authorize it if you really push, but you have to talk to someone with expertise about fraud risk, and have to spend some time doing it (which helps a lot with the "urgency" dimension of scams). Most permission approval prompts on phones/computers have no such human intervention or time-delay option, even if you might want them to.

      > The only way to 100% prevent scams is to remove the potential entirely. There are scams, infinite actually, right now, on iPhones. Do you support that? Surely not, so we should lock iPhones down more no?

      Nobody here has the goal of preventing 100% of scams. The ideal amount of fraud on these platforms is not zero (https://www.bitsaboutmoney.com/archive/optimal-amount-of-fra...). Rather, fraud is still extremely common and difficult to disincentivize. We want to reduce that as much as possible while not imposing overly-onerous restrictions. The discussion is about what counts as "as much as possible" or "overly-onerous", not whether we should ban phones or banks.

      > I can't install open-source software I've audited myself on my iPhone

      I hope you understand you're in a very tiny minority of people who can do that, using a device designed for people who cannot do that, and whose behavior if permitted to do what you're after has a proven history of causing significant damage to individuals (who lose their savings) and shared resources (sites DDoSed by end-user-device malware, hospitals that can't provide care because someone let ransomware onto a computer, and so on).

      1 reply →