Comment by saagarjha
5 years ago
It’s intensely frustrating, because for some reason Apple thinks it’s a good idea to strip out security code from the source that they do release (months late), and they tend to strip (and until recently, encrypt) kernel code. This is what a company from the last decade might do to hide security issues, except it’s coming from the world’s largest company with a highly skilled security team. Is there some old-school manager with so much influence that they’re able to override any calls from internal and external sources? It’s gotten to the point where Apple engineers privately brag about their new proprietary security mitigations after researchers who scrounge for accidentally symbolicated kernels (thank you, iOS 14 beta) do the work to find them. Why does this situation exist?
There were some Hacker News threads the other day about Marcan's Patreon campaign for porting Linux to Apple Silicon. Everyone basically expects that Marcan will need to reverse engineer everything on his own, and my gut tells me they're right.
But, if you actually stop and think about it for a moment... isn't this situation completely bizarre? Apple Silicon Macs explicitly support booting alternate OSs, because Apple went out of their way to add a `permissive-security` option to the boot-loader. They know Linux is important—the initial Apple Silicon reveal included a Linux VM demonstration—and now a well-known and talented developer is planning to do a native Linux port, at no cost to Apple, and we all fully expect that Apple won't make any documentation available or answer any questions? And, we're probably right?
The more I consider it, the more crazy it all seems. Why is Apple so private about the internals of their products? It won't affect marketing—normal consumers don't care—and I can't think of a plausible scenario where this type of information could help a competitor.
Is Apple using source code stolen from Oracle? Are they scared someone will discover an internal library written in COBOL and make fun of them? Are they worried their documentation could revive Steve Jobs as a vengeful ghost? I just don't get it.
> Why is Apple so private about the internals of their products?
Because they don't care. The extent they care is directly linked to the amount of money they will make from caring. They won't sell more macs if macs can run Linux better; but they will sell more Apple Music subscriptions if macs keep running macOS.
> They know Linux is important
No, they know Linux is a pain in the ass. The bootloader option assuages the executives' conscience enough to be able to talk to a journalist and keep a straight face when asked about "openness" or being "hacker-friendly", stuff those 1980s-style Linux hobbyists keep talking about and nobody else gives a shit about.
Apple makes money by selling iDevices to consumers and selling Macs to enough developers to build apps for iDevices. Everything else is a bonus, and not worth spending much time on. They do the minimum and leave it as that. There is no inconsistency or secret motive. They just don't care. When they cared, in the early '00s, they did a bit more; now they do less. The attitude is the same.
> Apple makes money by selling iDevices to consumers
For now. They haven't made any "I have to buy this right fucking now!" worthy revolutionary improvements in the iDevices lineup recently, the Western market for smartphones is near saturation - and with Corona tanking the US economy for wide masses, people don't have the hundreds of dollars just lying around to shell off for the latest iteration.
I believe that both the last (horribly expensive at that, and still people kept buying it) Mac Pro and the new M1 lineup is a sign that Apple wants to shift back attention to the non-mobile sector - because the competition there is asleep on the wheels. Everyone uses Intel who has managed to fuck up its lineup for many years now, Microsoft has thoroughly pissed off the privacy-conscious folks with Win10 and (judging by a random walk through a Mediamarkt) build quality in Windows laptops still hovers barely above "acceptable" - cheap plastics, tiny touchpads and abysmal screens are the norm, whereas Apple is only robust aluminium cases, giant touchpads and crystal clear, bright screens.
What I'm really excited for is when Apple decides to put an Mx chip into an iMac, paired with a decent AMD GPU. The thermals and energy profile should be allowing a lot more leeway for resource usage than a Macbook...
> they will sell more Apple Music subscriptions if macs keep running macOS.
The type of person who buys an Apple Silicon Mac to run Linux is not going to buy an Apple Silicon Mac to run macOS. However...
> They won't sell more macs if macs can run Linux better.
They would sell some more Macs. Possibly hundreds of thousands more. A drop in the bucket for Apple, but still money—and all they have to do to get it is answer some questions.
Even without being able to compile it I've successfully used their source dumps to debug problems in my code quite a few times (and occasionally find bugs in their code which I have to work around). Having code with comments to read is a huge step up from having to rely on decompilers.
(Quick note for others that my GP comment originally contained a paragraph about the stuff Apple does open source. I edited this out because I felt it was beside the point.)
> P.S. And what's with the stuff Apple does release as open source? Don't get me wrong, I'm glad they do it—because I'll take what I can get—but I have no clue who it's for! A lot of the code is either extremely difficult or impossible to actually compile, because it relies on internal Apple tools or libraries which aren't public
Even when it doesn't rely on anything Apple-specific, it can be unclear how to build it.
I noticed that if I ctrl-z dc, then resume it, it silently exits. I grabbed the source to see if I could build it, and then perhaps debug this.
The source is part of bc. When you extract it there is a directory containing a bc dir, a patches dir, a bc sources tarball, and a Makefile. The bc directory is the contents of the tarball with the patches from the patches directory applied.
Optimistically typing "make" does not work. It runs configure somewhere (in the bc directory, I think), decides that gcc is /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc, and decides that this cannot create executables and exits.
Maybe just going into the bc directory and running configure and make there will do the trick? ./configure works and builds a makefile. Trying to compile with that gets fatal errors, apparently due to missing <string.h> in some C files.
OK, I don't actually care about bc, so how about just trying to build dc, which lives in a subdirectory under the bc directory.
That gets a fatal error due to a conflict between "#define ptrdiff_t size_t" in the config.h that configure made, and "typedef __darwin_size_t size_t" from somewhere. Based on the comments in config.h, apparently it should only be defining that if it is not defined by the system. Commenting it out in config.h and trying again...and all the compiling steps for dc actually finish!
Alas...it then fails because it needs ../lib/libbc.a, which presumably would have been built before building dc if the bc build had worked.
Maybe if I go to ../lib and type make? Nope. In fact, the errors are identical to when I typed make for bc, because it turns out that making libbc.a is the first thing the bc make tries to do.
Tossing in "#include <string.h>" in lib/getopt.c and lib/number.c makes everything build, finally giving me a locally built dc.
Is it too much to ask that when I download the source from Apple to their version of a simple command line knows-nothing-about-MacOS utility like this, I should just be able to type "make" somewhere and have it build? Or at least have a README in the tarball that tells me what I need to do?
In this case, the top-level Makefile includes a bunch of internal junk, and the configure script thinks your system is very broken because it's old and Xcode 12 ups the warning for a missing prototype to an error. I was able to get it to build with
Lots of the stuff they release I imagine is to comply with license obligations.
In a few cases perhaps—they definitely still use some GPLv2 stuff—but it's mostly under a license that doesn't require them to release anything.
I can only speculate, but Apple seems to have very tightly coupled software and hardware. Since this coupling probably holds trade secrets (which we don't know about by definition), it seems likely to me that they are controlling access to as much of the stack as they can while still protecting those secrets.
Yes, but that doesn’t really make sense for things they have already shipped: researchers have to reverse engineer those for what seems like no reason. For example, the newest iPhones have entirely custom privilege levels that are lateral to the typical ARM exception levels and entered using proprietary instructions that their own silicon understands. This is something you can find if you load the kernel into a disassembler and poke at it a bit. But Apple doesn’t mention it at all or document it…what’s the point? Why put up such petty barriers in the face of people trying to audit this?
Likely the documentation that does exist internal would take a relatively large amount of cost to extract without pulling other stuff with it.
4 replies →
Because you're missing the other half of the exploit market: Selling vulnerabilities for big cheques.
https://zerodium.com/program.html
I'm not sure how that applies to my points?
> Apple thinks it’s a good idea to strip out security code from the source...
Because it makes it easier for attackers to find vulns. That's why they do it. The high payouts on these platforms are evidence that it isn't as simplistic as "only defenders would look at this why won't they release it!?"
But doesn't it work in some ways? It's not going to save them, but it seems to significantly increase the time/cost of exploiting the vulnerability. One more layer to the security system.
Obfuscating source? No, not at all. It just annoys legitimate security researchers (making them not want to deal with you) and is something that black hat bug finders largely don't care much about. Not only do they have more resources and patience, they are also more willing to use questionable methods to make their lives easier.
What makes it less of an issue for black hats? Do they have access to symbols/source code that security researchers do not/are not willing to use?
I certainly understand the frustration for legitimate researchers, and there's plenty to be said about having the source code available to make auditing easier but in itself it seems that making a black hat take 6 months instead of 1 to create an exploit raise the skill/patience level needed and busy them for while where they are not working on the next exploit.
2 replies →