Interoperability can save the open web

3 years ago (spectrum.ieee.org)

A smart, insightful interview of Doctorow, with a good interviewer, Michael Nolan. Reasonably optimistic, the framing is on target, dare I say inspiring.

Read it; it is worth your time. That's more useful than dwelling on thoughts such as "the Web is past its prime" and "even if we improve it, Big Tech will exploit it". Repeating the negative generalizations over and over can contribute to a self-fulfilling prophecy.

If so inclined, after you read it, find a way (small or otherwise) to help.

  • I did just that, years ago. I took a bet on the open internet.

    I don't have to convince big tech to treat me right, there's already a whole big open internet where I can go and what I can connect to. Existing techniques from the 90's and before still work. Websites still work. Websites without third party code do work. Analytics from Google? Thumbs from Facebook? One can remove that with ease. Just to name a few.

    So I ended up not with the most innovative approach (no blockchain, no NFT, no AI), but with a solid piece of software to build your own website with. One can connect to other independent websites via RSS. One can download it for selfhosting, one can take a subscription. You can design the thing yourself without knowing anything about design, all built-in functionality. Use it for business with a webshop, as a personal blog or photobook, or as an organization with an online info library and custom forms to connect stakeholders, etc. Oh, and visitor statistics without cookies and tracking.

    Websites don't necessarily have to stay websites, they can evolve into personal online multi-tools that work nicely together with the rest of the open web.

    My main concern - broadly speaking - is knowing that the future content of every website I deliver might end up 'raped' by some AI company and 'abused' as part of their product. But still, it doesn't stop me from betting on the open internet, because AI products don't exclude me and my customers from publishing our content on the internet.

    • I followed the link and gotta say, love the approach you took.

      About AI, totally. I guess my hope is big publishers waking up and realizing they are too being ripped off by big SV tech that's just blatantly ignoring copyright.

      3 replies →

  • >> Repeating the negative generalisations over and over can contribute to a self-fulfilling prophecy.

    Very true, but it's also treacherous to go blindly in the opposite direction. We need to be capable of both modes of thought... even if they contradict occasionally.

    Interoperability definitely has the potential to "save." It's very good to have someone focus in on that with optimism and conviction. OTOH, we also need to remember that there are (or at least may be) other factors at play.

    At this point, "freeing the web" likely entails bankrupting FB and Google. Their ad businesses just do not work without dominant market share, control over user data and such. See Twitter, Bing, reddit, etc. They don't have pro rata earnings relative to the big boys.

    Can Google and Facebook fail, without causing disaster, political mitigation efforts, recession, etc? I'm not saying this to contradict the arguments, just pondering the scale of the task at hand.

    I think a usable middle road might be to focus on interoperability's direct, first order achievables... not the big picture. What, in the most practical and down to earth terms, can be achieved by and achievable step in the interoperability direction.

    • Agreed. I'd put the emphasis on avoiding dwelling and maximizing detailed system awareness with a bias towards doing something.

It's the only thing that can. Interoperability should be written into law and stuff like remote attestation should banned. Otherwise there will be no such thing as "open" anything. There is no freedom if they refuse to interoperate with you for daring to exercise it.

  • Indeed, you have to go legal about this (what I am exactly doing, but my lawyer gave birth then it takes some time).

    "Interop" alone does not mean that much anything: what Big Tech is scared of, small software, simple protocols, able to do a good enough job, which it is "easy" and "cheap" to develop an alternative of.

    For instance, IRC(TLS) bridges, noscript/basic (x)html (HTML forms can do wonders). EU started to emit personal user certificates for auto authentication, and let me tell you: it is HORRIBLE to install such certificate in Big Tech browsers... and I am suspicious about the certificate file format (never really got into it).

    But don't fool yourself, Big Tech "knows" and will fight it, then expect the worse: they will shadow-hire teams of hackers to destroy your alternative. The part of having up and properly running "juicy" public internet servers up is 50% of the job... and it gets worse if you have a payment processor.

    • > Indeed, you have to go legal about this (what I am exactly doing

      That's extremely interesting. Can you tell us more?

      > "Interop" alone does not mean that much anything

      True.

      By "interoperability" I mean being able to have any client connect to any server without discrimination. They should not know or care what software I'm running, only that it speaks the same network protocol. Remote attestation violates this by enumerating approved clients and cryptographically vefifying them.

      "Adversarial interoperability" is an even more interesting concept.

      https://www.eff.org/deeplinks/2019/10/adversarial-interopera...

      > But don't fool yourself, Big Tech "knows" and will fight it, then expect the worse: they will shadow-hire teams of hackers to destroy your alternative.

      I don't expect it to be easy. We're talking about amoral trillion dollar corporations who only work for their bottom lines. I wouldn't be surprised if they killed people over stuff like this. Coca-Cola did.

      2 replies →

Not sure when this article was prepared, but it doesn't mention WEI and other related technologies (remote attestation) which is IMHO the biggest current threat to interoperability. Sure, the "standard" is open, and theoretically anyone can implement it, but when the only "trusted" keys are those owned by Big Tech, it's hard to argue that's actually open.

  • The biggest threat against the open web are the malicious actors on it like spammers or fraudsters.

    • Spammers and fraudster are a problem, sure, but the biggest? By what measure?

      If we're trying to predict which threats are "big" enough to lead to system failure, the analysis is quite different. In natural systems, parasites tend to fill niches and can persist for lengthy periods, often as long as the host. Or longer.

      Think of it as a historically situated evolutionary battle. Thinking over many scenarios, there are many failure modes. One way to tease apart the likely causal threats involves thinking through a lot of scenarios.

      Under what conditions you think spam/fraud would (more or less) 'destroy' the open web? And what does that destruction look like to you?

      1 reply →

    • The biggest threat against the open web is Google who wants to turn the web into yet another AOL with vertical integration.

      It is the corporation who wants to put malware on your computer under pretense of "verifying content integrity", just to force people to see their ads from spammers and fraudsters.

      Google ads often promote scams in Google search.

... and other fairytales we tell ourselves

The web is dependent on big tech, so interoperability will always be limited by shareholder interests

We have to consider the alternative hypothesis, that interconnectivity opens up markets and leads to global competition and winner-take-all phenomena, whether it is the internet, the money market, export controls, foreign real estate etc etc. If we really want the small guy to win again, maybe we should wish for the opposite

  • > The web is dependent on big tech,

    If that were the case, it would never have been built until big tech came along and invented it.

    As the web existed for years before big tech got involved, and for a few more years after they got involved but were still playing catch-up, I don't see how that holds up.

    > interoperability will always be limited by shareholder interests

    Again, history shows the opposite.

    There were a number of attempts by large corps to create something like the web before the web, like Minitel and Compuserve, and to some extent AOL. (Also see X.400 messaging before SMTP email) They all lost out to the web because they had gatekeepers and no interoperability, whereas the web with its interoperable standards allowed anyone to come along and build anything.

    The more you tighten you grip, Tarkin...

    • In a sense, it is dependent on big tech. Intel, AMD, et. al make the substrate upon which all of tech is dependent.

      We don't notice it so much because the era of monopolistic dominance ended 30 years ago when AMD cloned the 386. The ability to write code that didn't utterly depend on Intel came along strong in the years since.

      I think we're in that era now. People are beginning to notice that "social media" isn't really. People just want to send messages to friends and family, share photos with friends and family, and maybe catch up on what somebody famous, like Richard Stallman, is doing. (I have my finger on the pulse of public taste doncha know.)

  • The web's interoperability today is almost a miracle.

    Take a look at other industries. Proprietary formats everywhere. You want to make music? Ok buy these VST and AU. Make games? FBX and PSD go burrrrr. And I'm sure it's only getting worse at more traditional, B2B industries.

    • Are you really sure of this? I can take any detergent for my dishwasher, any pots for my stove, plug any device into the sockets on my walls. I can continue this into eternity, you get the idea.

      There are so many industries that really rely on interoperability and have been for so long, it’s so natural to you you don’t even notice it.

      So, going back to your argument: it’s like Doctorov says - the phase of immense oligopolies right now is not the natural state of the web, and something we’ve seen with other industries in the past. We can do something about this.

      1 reply →

    • > Ok buy these VST and AU.

      That is why CLAP, a open spec is being pushed by some DAW. Steinberg(Yamaha) literally forbids to distribute older VST SDK today. The VST format is not open. CLAP doesn't have that problem.

    • Yeah. I think we should protect this miracle at all costs. It's too good to just sit and watch it be destroyed.

Unfortunately, the Open Web just makes it easier for Google to smart snippet all the things. And it's not even about monetization (via ads) for the source content provider, but how about giving some credit to the source.

  • Having Google crawl everything on an Open Web is immensely preferable to the alternative of a closed web, or no web at all. Part of uploading things to the internet is reconciling that everyone can see, copy and distribute the content you provide. It's part of authoring anything digitally, and a poor boogeyman in a world where the Open Web has few demonstrable harms.

    What should really scare people is the prospect of a common interface like the internet disappearing and being monetized by private interests. We take the Open Web today for granted, and while I partially feel like Doctorow is too fatalist, I also agree that interoperability is a core part of what makes the web function.

    • Even if that is true, that is not what users want. They do not want everything they have ever posted to be out on the internet with no way to delete it.

      Scrapers are hostile actors against users which is one reason social media like sites invest resources to defend their users against scrapers.

      2 replies →

    • >> Part of uploading things to the internet is reconciling that everyone can see, copy and distribute the content you provide

      In some ways this is a narrow definition of the Web. There is a lot of activity placed behind a login to expressly prevent the information from being public access.

      If I upload a private repo to github I expect it to be private. If I interact with a Dr or lawyer on a site, I expect that to be private.

      Of course inter-operability controlled by the -user- is different to the idea of interoperability controlled by the host, or by some external entity (scraper). The former is good, the latter less desired.

      1 reply →

  • Interoperability isn't the problem. Leverage to enforce your own IP, or lack thereof as an individual, is.

    Just because you publish content on the Web doesn't mean you give license to anyone to use it however they want. IP is rooted on a foundational principle of giving explicit consent. Copyleft is using that principle to explicitly state "anyone is free to use this however they want". Without that consent, it's assumed that the author can ask you to cease and desist. (Hence why e.g. wikipedia is plastered with creative Commons license mentions)

    Sure, there are fair use exceptions. But if you take a close look at the conditions that need to be met before a published copy can be considered fair use, it's not as clear cut as it seems.

    Thing is, only big media outlets with capital, like the NYTimes, are able to litigate against big actors who wholesale misuse interoperability after a tragedy of the commons kind of fashion.

    This imbalance in resources and capital to enforce rights between a handful of big actors and everyone else is exactly what Doctorow draws attention to in the interview.

  • > the Open Web just makes it easier for Google to smart snippet all the things

    How is this a problem? A simple `Disallow: Google` would solve your concern if you want to de-list from Google.

  • Does Google have a track record of not respecting robots.txt? Otherwise why is it a problem?

It is a bit unclear to me what exactly is being proposed. That every app exposes some kind of common standard API that can be used to link with other apps? Take their twitter example. How would “taking your followers with you” work in practice? I can imagine some sort of indirection layer that associates each user with their respective provenance domain + an update protocol if the provenance changes, but there are likely challenges when it comes to scalability. Who is going to design/mandate these interfaces and where will they be applicable? What’s the criterium?

  • In one word, MyData: I should be able to authorize a new service Y to download all data about me from service X. I should be able to delete all data about me from service X and leave a redirect to my new profile URL at service Y.

    Number portability is not a novelty requirement in competition law, but somehow we haven't expanded it to apply to online accounts.

    Standardisation is needed but that's why RFCs, W3C etc. exist. The existing web standards go a long way if fully implemented.

    • Thank you! I totally agree that data should belong to the user (in fact, I find it shocking that some consider this opinion controversial). I’m still not quite clear however how the software can interoperability is supposed to be achieved. I understand that one can impose a standard, but a standard presupposes a certain data model. What if my application does not follow this data model? Would one ban social network apps that don’t want to implement a particular database schemas?

      2 replies →

  • The exact example you're describing is what the ActivityPub spec and the resulting "Fediverse" are aiming to solve. Mastodon and compatible microblogging platforms are distributed versions of Twitter, Lemmy & Kbin are compatible distributed versions of Reddit/HN style link aggregators.

    Given the boost both of these parts of the fediverse have gotten from Twitter/Reddit's series of missteps recently, they're starting to look like they could become viable long-term alternatives.

For VOD platforms. it would be a dream if one day netflix / disney+ / AppleTV / ..., would offer iframes, and I could use other sites that offer me their own catalog browsing and recommendation ergonomics that are not limited to a catalog with their walled garden editorial choices.

Why have iframes lost so much use?

  • Bunch of issues:

    - security (now improved but historically very problematic)

    - accessibility (confusing for screen readers)

    - responsiveness

    - scrolling

    - usability (for example if something in the iframe wants to display modal with backdrop covering whole screen)

    But I agree that Netflix catalog UI sucks

I agree with the premise of the article. Part of me thinks that a lot of the drama around data privacy was manufactured by big tech precisely to get people riled up about privacy protection in order to ensure that such interoperability solutions (which are ultimately about sharing data) would not see the light of day.

But this is casting a shadow on the idea that some users may actually want to share some of their data broadly and across platforms in an open and interoperable way. Regulators were so busy focusing on data protection, they completely neglected the other side of the coin which is data propagation. As a user, just as I have the right to have some of my data protected, I should have the right to have some my data freely propagated. Kind of like copyright versus copyleft in the software industry.

It seems like it should be possible to force companies which have a monopoly (or near-monopoly) to at least allow users to opt in to data sharing. Like for example, there should be a way for me to tell Facebook to share my name and/or email address publicly via API...

Though on the flip side, I think Facebook should be allowed to delete my account (with appropriate notice) if they don't want to support making my data public since they shouldn't be forced to bear the bandwidth costs. In any case, I think this would offer people with the option to confront big tech platforms about appropriate use for their data. People own their data and should be able to set the terms and change the terms any time they want. Big platforms with a near-monopoly should not be able to make it an all-or-nothing (if you don't like it, delete your account) kind of deal.

  • This is a difficult topic. On one side, it seems wrong to force platforms to pay for hosting and bandwidth costs associated with bots scraping their open user data... Yet it also feels wrong for big tech to continue to use their monopoly position to take away users' bargaining power in terms of control over their own data.

    It feels like the free market solution would be to form some kind of large group, like a union, a syndicate whose members would agree to delete their Facebook (or whatever platform) accounts en mass unless their demands are met (similar to how a union works when employees go on strike). Then Facebook (or whatever platform) could decide whether it makes sense to lose all these users/accounts or comply with their demands to make their data public and take the costs of hosting/opening up that data. Then this would not require government intervention.

    I think one thing the government could do would be to facilitate the creation of such group/syndicate via a large advertising campaign. That seems like the right level of government involvement.

    • The union/syndicate idea is interesting, but as with unions, reprisals are possible since there is little to no anonymity on social media, and no legal protection. Reprisals in the form of affecting your work, for instance. Not just affecting your access to social media, but these conglomerates offer other goods and services, and maybe they'll just update their EULAs so that such actions forbid you from using their other services...

This solution effectively comes down to "get regulators to police big tech more, in this particular way".

How much luck have we had with meaningful regulation in the tech industry recently? Doesn't feel like much, if any. Everyone hates GDPR and the cookie banners. Restriction of advertiser tracking — the closest thing to a consumer win that comes to mind — was not a product of policymaking but of Apple's competitive action.

I personally love the idea of government regulation but America clearly doesn't. This market needs solutions that don't involve top-down control.

>Doctorow proposes forcing interoperability

No surprise that Doctorow proposes authoritarian solutions to create an "open web".

"It's open unless you run your server in ways me and my fellow ideologues disagree with"

  • Curious what your thoughts are on the paradox of tolerence? Google and the person hosting a static blog on a server in their basement are not the same.

    • Legal consequences, which are backed by the state's apparatus of violence comprising of police, judges and prisons, should only be used against those who engage in fraud or violence. Running a closed API, as much as I may dislike it, is neither of those things.

      There are plenty of other ways to counter closed APIs that do not rely on initiating force.

      2 replies →

> Cory Doctorow presents a strong case for disrupting Big Tech ... Doctorow proposes forcing interoperability

Forcing? Like, legally requiring every media company to maintain an RSS feed and support activity pub? Also, is this what disrupting means?

  • > Forcing? Like, legally requiring every media company to maintain an RSS feed and support activity pub?

    What other way is there? Waiting, until they do it out of pity?

    The EU was playing around with the idea already, they just went a little overboard in their first drafts of the Digital Markets Act. But I think we'll see something like that come out of Europe soon.

    • > What other way is there? Waiting, until they do it out of pity?

      Did we legally force browser vendors to support web protocols and standards? Didn't the market solve this for us — i.e. if you created a browser that didn't do http, or https, or web sockets, or support a video or an audio html tag, then it would just die, because people won't use it. Are we admitting here that interoperability does not bring any competitive advantage, and that you need a state regulator to force you to do it?

      2 replies →