Comment by CharlesColeman
7 years ago
> OS Application Binary Interface (ABI) release inter-compatibility is the cancer killing the modern operating system.
I think that's only true perspective of an OS developer:
ABI inter-compatibility (e.g. the Windows and Linux model) prioritizes customer experience. Customers hate it when their applications stop working, and application developers don't want to spend lots of effort to track platform API changes just to avoid breakage.
Abandoning ABI inter-compatibility (OpenBSD, Apple) prioritizes platform developer experience. They want to be able to freely make API changes and don't want to spend time maintaining old APIs that they could use to work on new APIs.
I think the problem with the latter is, while there may be a few hundred developers working on a particular OS, there's are orders of magnitude more customers and application developers. Totally abandoning ABI inter-compatibility seems like putting the interests of the very few over the interests of the very many.
> I think that's only true perspective of an OS developer
I'm not really an OS developer* but I feel like all the build up of clutter and deprecated-but-still-there-but-unmaintained dead-end APIs with kinda-working (with caveats) replacements and redundant tools (targetting old and new APIs) is actually significantly hurting my experience as a user as well as a developer who develops applications on top of the platform with all that clutter. That's right, I hate Linux as a development platform as well as in terms of UX. By contrast, the Linux kernel, where they do not maintain internal API compatibility, is a much nicer area to be writing code in.
I love OpenBSD specifically as a development platform as well as due to its UX. (Not all of its qualities can be attributed to lack of legacy clutter and ABI stability, but I do believe it plays a role)
* For day job, I develop and maintain low level systems software on a custom embedded distro, along with some kernel bits.
Both, as a user and a developer, I am very happy, that Microsoft 10 still ships with ActiveScripting, ActiveX and COM, as well as all the Win32 stuff. Because, anything, that came after that, was a pile of crap for power-users. Of course, these are user-space technologies.
Naturally it still ships with COM, that is what Windows modern ABI is all about, since Longhorn.
It's important to remember that this works for OpenBSD in a context where a lot of the tools it's used with are in base and developed in sync with the ABI, by the same developers, or in the ports tree, which is maintained by an extraordinarily capable team of generous volunteers. It's not zero-effort, by far, but the effort is also not offloaded to its users. The people who break it are also smart and responsible enough to also fix it, so the breakage users see is basically zero. (I'm using OpenBSD on some machines, and I've been bit by this once or twice, I think, since OpenBSD 3.1, which was released in 2002).
Transplant this approach to Windows, where users habitually expect software that was built twenty years ago against Windows XP to still run, and you get a lot of angry people calling.
IMHO (spoiler alert: OS developer, but not on OpenBSD), whether or not you want to spend time maintaining old APIs shouldn't be relevant, not after you reach adulthood anyway. I don't think OpenBSD chose this approach because the alternative just ain't fun. If a stable API is part of your approach, then that's what you do. It's not glamorous but a lot of things in programming aren't. You can certainly add new functionality while maintaining compatibility. Linux does it pretty well, for example. (Linux does break things now and then, but very rarely.)
It's also a valid choice to not maintain ABI compatibility or API compatibility, but making that choice doesn't magically absolve you of making this work for users and third-party developers.
There are projects that eschew this responsibility and offer various silly reasons for it. Unfortunately, unless you own and operate the kind of reality distortion field device that Steve Jobs owned and operated, people usually see right through the silly reasons and tend to lose patience with this model (unless they're financially committed, which is why it doesn't work that well for FOSS software).
Except it means unbounded growth in complexity, which inevitably leads to, among other things,
- More bugs
- More vulnerabilities
- Increasing cost of support
Which absolutely impact the customer experience, just not in the short-term. A little bit of short-term friction averts long-term intractability.
That's true: those things do negatively affect customer experience. However they don't affect customer experience as negatively as the experience of having their software break.
To put this in another context: newer building codes may result in better and safer homes, but it'd be extremely user hostile to force homeowners proactively upgrade their homes to compliance each time a new version is released (at the threat of having their home condemned if they do not). The sensible compromise, in buildings and software, is to allow things to be upgraded over time, as they're modified.
From the OP: > declining concern for the future in comparison to the present
Breaking backwards compatibility has a larger negative impact in the _present_ than the cruft of old APIs and code. But that impact is temporary. However, the negative impacts of cruft can have a larger impact _in total_ over the entire lifetime of the operating system.
> However they don't affect customer experience as negatively as the experience of having their software break.
They affect it far worse, because they affect _every_ user. Having unmaintained/outdated software break only affects the subset of users that want to use that particular software.
10 replies →
> A little bit of short-term friction averts long-term intractability.
It's not a bit of short-term friction, it's constant friction. As long as development is continuing, there's always something about the API that can be improved and would really pay off if only that was actually its final state and not just the next step until we find a good reason to break it again.
Except it may or may not be a only little bit of friction, depending on the breakage, and it is not short-term if it happens regularly.
Right - maintaining backwards compatibility is deciding to take on additional technical debt.
I think the implicit calculation here is that if you push a release that breaks the user's workflow, they can point to a specific point in time where things became frustrating and there will be a PR hit at that moment in time.
If instead you maintain compatibility, the small costs of all the technical debt accrue over time to make the experience worse than it might otherwise be, but users may not even notice or have a conception of what they may be missing for having stayed on this path.
They ultimately may end up with a worse product / UX, but they have no specific reason to complain about it.
You forgot lower performance.
"among other things" :P
I think by "customer" you mean "a certain kind of customer." Apple prioritizes their customers, who largely buy one device at a time and use recent, actively developed software, as opposed to customers who buy software licenses in bulk and sometimes need to run decade-old unsupported software. In the courses of this, they've attracted a lot of enterprise customers as well, but those customers aren't buying desktop software, because they're aware of Apple's development model.
Do Apple customers use recent actively developed software because they want to or because Apple often breaks their APIs and actively developed software is all that you can use?
I think it's both. Ten-year-old enterprise desktop applications may be functional and even highly usable, but they look and feel ancient. It's been a long time, but I remember using Windows software in the 2000s where I could almost smell the mold and dust. Applications like that are the software equivalent of Miss Havisham's wedding dress. If you're positioning yourself as a fashion-conscious brand selling products that become part of a person's identity, it helps to have a mechanism for sloughing off applications that aren't constantly being refreshed and rewritten. Constant rewrites aren't economical for enterprise applications with a few hundred or few thousand users, but those have moved to the web anyway.
Note that this doesn't reflect my personal values about software, but I can see how it serves Apple's priorities and keeps their products attractive in the eyes of their customers.
A contributing factor could also be that because for many in the Apple world, hardware goes in tandem with the software, it follows the natural cycle of hardware refresh. Sure every now and then someone finds that an app no longer works, but if they aren't paying big big bucks, who cares?
If you are writing applications that directly call OS ABIs you are doing it wrong.
In the OpenBSD and Apple worlds, you can change an ABI because applications are expected to, as they should, call libc and similar platform abstraction libraries instead, and these libraries are dynamically linked.
It’s not just those two - probably every mature system on the planet except Linux defines the system interface at the library level, not the syscall level.
By that definition Windows is freely changing ABI too, since the only defined interfaces are libraries like the KernelBase.dll or netapi32.dll.
But that only shifts the maintenance burden. Anytime Windows changes its inner working that means that every existing library has to be changed to expose the same interface to old code, including being bug-compatible and supporting all the abuses of undocumented features (or at least those used in software used by relevant clients).
I see this discussion about the conflict between user needs and developer needs coming up time and time again. I think this disconnect between user and developer is an interesting phenomenon in itself. What's more, often this disconnect seems to be defended as a natural feature of the service sector: it's an exchange of money for service, so the service provider's experience doesn't matter, as long as he's willing to work for that much. Reducing the relationship to money is often how these discussions go, in my experience, and I think that it's something that can look much better on paper (unintentional pun) than in practice.
Here's an example from my experience with ordering food or drinks. I'm floating the idea that it generalises to other areas as well, but that's up for debate. If I don't know the restaurant or bar well and I have a chance to ask the kitchen staff or bartender (in case of a drink) for suggestions, I'll do it. I'll ask what they like to make. I'll always ask indirectly, because it's a personal question. I'll engage them in a short dialogue in which I get a feel for _what would feel good for them to make_, _how_, etc., and then I'll ask them to do it for me. It's often something that involves a bit more skill or know-how or is stimulating to do in some other way. Sometimes it's something gourmand that they're proud to be able offer, and they're thrilled that someone's willing to walk off the beaten path and that they can accomodate that.
That's the geometric opposite of looking at the menu and picking what you fancy the most. In a way it's about not taking your likes and dislikes seriously, so as to stay open, because, this is a place you don't know, while, this person you're talking to, he essentialy is the place.
So why do I prioritize the experience of the service provider over mine? I don't. I recognize that my experience is very closely tied to that of the cook or the bartender, and if she's happy, all other things being equal, my chances to be satisfied are the highest. A happy cook makes better food, a happy bartender makes a better drink, and the experience of us aligning our needs leaves us both feeling uplifted, because this wasn't just another impersonal money-for-goods transaction.
Would you extend your experience with the food sector to your car? Say you bought a car that was made by people who loved working with it. The mechanics even love to repair it.
But now, 1 year passes and you go for a regular check-up to your mechanic and they start visibly sighing when you enter. They now hate repairing this ancient piece of tech, it barely matches any of their tools, and they caution you that the tool manufacturers are actually moving to a six-month schedule for new car repair tools getting released, and half the tools they use for your car are already deprecated and likely to be discontinued next release.
Would you be happy to just buy a new car, to prioritize the service provider's experience? Or would you seek a different service provider/car brand, that doesn't do this?
That's a thought evoking question. I don't own a car, so this is hypothetical. Use a car that your mechanic does like to work on (doesn't start to dislike it after 1 year). In your scenario the tooling sounds like a significant investment, so the mechanic probably also "sighs" that he has to get new tools often, and that he can't service older models. Is there a repair-friendly car on the market that's also good enough in other respects? From what I hear people take note of repairability when choosing a car (I do it when choosing a laptop). If the mechanic liked the "1-year car" initially, he misjudged it.
I guess what I'm talking about is looking for win-win situations. I get the feeling that when it seems like you can't get there, a more fundamental problem has been introduced earlier in the process. Your example looks a bit complicated, because it's more obvious that the relationship is actually made up of more than 2 parties. Those explicit in your example: manufacturer, mechanic, driver. You could keep adding parties: regulator, importer, suppliers to factory, trade unions, etc.
That is not what it looks like at regular car repair shops.
What really is about here: the selling point of OS is software running there (which was proven by windows) The more it is a moving target, the more costs are on developers part to support old versions and follow the new OS releases. And the less software is going to support it.
As a developer I love everything that is new (I mean NEW not repacked old technology!), as someone that has to also support software on system level, I absolutely hate it.
I actually think the opposite is true. Typically customers choose MacOS over Windows (when price is no object). The features in the Darwin/XNU systems are certainly very customer focused. Having said that anecdotally I think that while it could be said that developers enjoy having new capabilities rather it is more often the case than not that when old APIs are deprecated and new APIs are used to replace them (forcibly) developers end up upset (see: Metal replacing OpenGL on MacOS / 32-bit application deprecation on MacOS).
> Typically customers choose MacOS over Windows (when price is no object).
But when they do so, I think they do it in spite of the lax attitude towards compatibility, due to other factors. Chief among them is the higher prestige Apple products have, the fact that many basic users only use very few programs that are all under heavy development (e.g. browsers) that lessens the sting, and many regular users have become resigned to being abused by their technology providers.
If you narrow the question down to one factor, we're going to break your stuff vs. we're going to do everything we can to keep your stuff working, I think most people would choose the latter
I think backward compatibility is a feature, and the lack of it can be considered an anti-feature as I mentioned in the article:
> As nice as backward compatibility is from a user convenience perspective when this feature comes as a result of a static kernel Application Binary Interface this trade-off is essentially indistinguishable from increasing time-preference (or in other words declining concern for the future in comparison to the present).
I would argue that users are not using the system due to the lack of backward compatibility, rather my contention would be that this feature comes at a cost that outweighs the benefit (also from the article):
> This can be seen with a continuous layering of hacky fixes, sloppily 'bolted on' feature additions (built in such a way that new additions don't conflict with existing APIs), and unremovable remnants of abandoned code segments left in place purely to ensure that applications continue to run. These issues not only cause their own problems but, in aggregate, cause a huge creep in the number of lines of code in privileged memory. This does not grow the exploitability of a system linearly but rather it causes exploitability to grow exponentially due to the fact that by there being more code to exploit, malicious functionalities can be chained together and made more harmful.
I am not upset because Metal replaced OpenGL. I actually like Metal's design.
What I am upset about is the removal of a standard that worked well. Just don't allow OpenGL apps in the Store and that would be enough.
Even worse, removal of 32-bit support in macOS. Now that's a extraordinarily bad move that confirms Apple does not care about enterprise nor gaming.
AFAIK OpenGL hasn't been removed, and isn't planned to be in the near future; it's just been deprecated. Apple isn't adopting new versions of OpenGL or Vulkan, but OpenGL code that used to run on macOS still runs on macOS.
5 replies →
> Even worse, removal of 32-bit support in macOS
32-bit macOS used a fragile ABI for Objective-C, meaning all ivars had to be in public headers and changing any of them changed the runtime layout of your class and _all_ subclasses. AKA adding a field was a breaking change. This imposed a huge maintenance burden up and down the stack. Often classes would include a void* pointer to a side-table where new fields could be added, or the public class was a wrapper around an internal implementation. Both of those have a performance cost (extra pointer chasing + extra mallocs, or double objc_msgSends for every method/property).
There are many other performance optimizations (eg non-pointer ISA) that were impossible on 32-bit, meaning every app that refuses to move to 32-bit imposes even more performance costs. Shipping 32-bit versions of frameworks also bloats the size of the OS and forces the OS to load a duplicate dyld shared cache. It even makes system updates and installs take twice as long (two shared caches need rebuilding after all!)
None of this accounts for future optimizations and improvements that would have been nearly impossible if the system still had to support 32-bit without forking half of the userspace frameworks... that kind of forking would be an absolute maintenance nightmare.
The deprecation of 32-bit has been obvious for over a decade. It is based on sound technical reasons. You may not agree with those reasons but they have nothing to do with not caring.
You might instead ask why a developer would choose to ship a 32-bit app/game anytime after 2010 when 64-bit was available and obviously the future? Should all Mac users continue to pay the cost in disk space, memory, and performance? How long should they pay that cost? Were you hoping macOS would support 32-bit applications in 2030? 2040? What about the undeveloped features and unfixed bugs due to the overhead of continuing to support 32-bit?
3 replies →
OpenGL has not been removed from macOS though. It’s just been deprecated.
1 reply →
There is a lot of weird stuff in Darwin that makes me wonder... Why do they deprecate useful things at other layers and keep this clunky Mach thing? They could switch to FreeBSD or Linux and probably perform better. It is hard to take them seriously when they say they don't like to maintain creaky stuff when there is Mach...
Sounds like you don’t understand what Mach actually offers. Check out Amit Singh’s book.
4 replies →
> Typically customers choose MacOS over Windows
Well, unless they want to play games or run any non-mac software … I don't know where you get your idea that customers prefer MacOS, but apart from some niche communities (designers, some subgroups of developers) this simply doesn't hold.
Only FOSS developers are upset with Metal, everyone else appreciates not having a clunky 3D API still based on C, and is already taking advantage of it on their 3D engine.
Plus, regarding the theme being discussed here, the use of Metal is transparent when using SceneKit, SpriteKit, Core Graphics,....
It is very much not the case that only open source developers prefer cross-platform standards to Metal and all the other vendor-specific APIs.
I have to say that I don't know why you dismiss open source graphics stacks. Anyone who has worked with the open source Mesa knows it is a breath of fresh air compared to the proprietary Qualcomm drivers, Mali drivers, Apple OpenGL drivers, or (horrors) fglrx.
5 replies →
Customers also choose Ferrari over Volkswagen, when price is no object.
RTFA:
Computer system design reflects the business that a company is in. It isn't the case that after years of development Microsoft has ended up with a bad operating system because people at Microsoft are idiots, rather it's the case that they're in the enterprise software business.
It isn't the case that Linux has not adopted the architectural advancements [...] because they aren't smart enough to implement those changes. The reason they have not adopted these changes is because they are in the business of ensuring that the people who pay them aren't made unhappy by a massive change to the kernel's architecture that necessitates a non-trivial expenditure of time and capital to modernize all it's software products just to keep them running.
Computers are becoming less secure and in many cases only a few systems are continually innovating in both the APIs they offer developers and the architecture of the underlying system itself.
I thought Linux didn't provide a stable ABI and tells developers to upstream instead? Is this the same topic?
The article is about OS ABI for applications (which Linux has), not internal OS ABI for drivers (which Linux hasn't).
Linux doesn't have it, but Linux Trebelized has it.