I skimmed through the article, but I have a question that I hope someone can answer. I have a sparse disk image created on a NAS (which runs Linux), and I use it to backup some stuff (not a VM) from the Mac in the native format (the macOS APFS file system).
Would this new format, ASIF, make this faster and better whenever I switch to macOS Tahoe? I hope there wouldn’t be any gotchas with respect to storing this disk image on a NAS.
Yeah I’m interested in this as well. There’s something about the way time machine works over SMB that is absolutely unfashionably dog-slow. I suspect SMB performance on mac is just not very good in general tbf
What blew my mind recently was that I could store an APFS sparsebundle on a NAS drive, then mount it over NFS and use it as a plain old APFS volume. Despite the filesystem layering, it works pretty much like a local volume, albeit with a bit of performance degradation. Seems preferable to something like iSCSI for using APFS with network storage.
Not sure why that would blow your mind, but Apple's old Time Capsule basically mounted a sparsebundle over the network too. And yes I would also guess this new format would work better.
My experience with NFS file management has been less than stellar, so running a full, virtualized, and performant APFS volume on top of it feels like a bit of a magic trick.
They also announced new APIs for native Linux containerization on MacOS with a specific focus on security and performance. This seems like it may be in support of that as well. Anything they can do to improve the performance of containers is a huge win. https://youtu.be/JvQtvbhtXmo?si=3OphClGvylHggmSW
I have a meta question not directly related to the article but more about HN itself. I posted this exact link 9h before this submission was posted[1]. How is it possible that there is a new entry for the submission given the link is the same?
In practice, if it gets any real amount of votes or comments, you have to wait a year to repost. If it doesn't get any attention, it can be reposted quickly (though I think it should be a day later).
It depends. I tried posting a few links I came across over the past few days, and it just showed me dups but they were from days or weeks or sometimes months ago.
My guess is that “fastest gun in the west” might be a bit anti-pattern with respect to community.
Because in ye olden times, mild URL shenanigans seemed to have been a common hack to bypass more strict dupe detection.
And the community probably doesn’t really benefit from aggressive karma seeking —back then being first would give you a point for every resubmission. [1]
But that’s all speculation based on a supposition that what is more likely to be submitted by other users is not a better criterion for choosing to submit.
But I could very well be wrong and probably am.
[1] and number of submissions is probably at best a noisy signal for front page placement and might be negatively correlated with curiosity…I mean even here, what Apple is doing doesn’t stray too far from yesterday’s big press release by one of the most valuable corporations in the world.
Might be something weird with this domain. Look at the list of submissions, yours isn't the first dupe that was accepted in a relatively short time window.
3rd party supporting a file system would be one of the last things on a list of all software I’d ever want a 3rd party writing instead of the OS maker.
Nightmare to evaluate the options, pure stress testing the options, difficult to know if it didn’t mess something up.
This article describes a new disk image format (on which a filesystem can be put, APFS in the article), not a filesystem, or did I misunderstand?
edit: added the word "image", which I apparently forgot to type. Mentioning the edition because otherwise it would make an answer to this comment difficult to understand.
ext4 and Btrfs are only well supported on Linux; they are not universal standards.
NTFS was only supported well on Windows until recently; but extensions like NTFS Encryption (BitLocker) are still Windows only. Mac still does not let you write to an NTFS volume.
APFS and HFS+ are obviously Apple file systems.
FreeBSD does not support ext4 or Btrfs well; but instead prefers UFS2 or ZFS despite also being an open-source Unix-inspired OS.
The world runs on proprietary or non-universal file systems with CDFS (ISO 9660), FAT, and exFAT being the sole exceptions.
Is there a single filesystem in the world (besides "simple" ones like FAT) that both has an open standard AND is licensed in a "usable" codebase (MIT or other "non-copyleft" license)?
Does anybody else think that it would make sense for Apple and Microsoft to just get in a room and horse trade a few things like this, if they cared about user experience? Cross-license both APFS and NTFS, and share any internal documentation under NDA so that external drives can use modern formats with safety features like journaling without locking users in.
I suspect there wouldn't be an agreement on a minimum set of features for a modern filesystem, even just for external disks, even if you limited it to flash storage devices to avoid all the complexities of spinning platter latency.
After all, there's no such agreement on Linux either - we just have all the Linux vendor options available.
This is a disk image format so why not VHD? It seems like it's open enough and supports what a virtual disk needs what do we gain with yet another file that's a disk type?
I think this is actually a human struggle - I add Conclusions to my blog posts / dev guides because it feels awkward to just end with the last step.
But it also always feels like I am just awkwardly restating things I just said. It's not a thesis, an article generally not complicated enough to need a re-synthesizing of main points at the end.
I have an unusual way of communicating sometimes that used to be quirky and fun. Now I simply get accused of being a bot. Before LLMs I never got accused of being a robot.
I think the idea of good writing is soon going to morph with writing that doesn’t feel like it was made with AI.
If you're going to be this superstitious, then you might as well lean into it and have a little fun. Better throw some herbs into the air and do a little jig for good measure. Personally, I'm going to accept that I won't be able to tell, because models are trained and prompted to appear human.
It's a shame that every new cool product/dataformat/cable/cpu/whatever researched by Apple has very little (or no) public documentation. Sure, there are lots of hackers who can test and reverse engineer those pretty quickly, but it's just unnecessary work. I don't know why Apple is so revered in hacker circles, to be honest. Not even Microsoft does this shit anymore, they're open sourcing a lot of research this decade, but they're still seen with extreme distrust. Whereas Apple was always secretive and used underhanded tactics, but it is still loved.
You're joking, right? You can see at a glance that this mess is only published for compliance reasons. There is no documentation at all, and most of the code consists of numerous versions of open-source libraries that have been subtly modified due to their licensing requirements, which necessitate disclosure of modifications. Good luck building any flavor of that! See: https://news.ycombinator.com/item?id=35197308
It's a solved problem in the sense that Apple solved it more than a decade ago. (The sparsebundle format was introduced in Mac OS X Leopard from 2007. The sparseimage format was even older; I saw discussions of using it with Panther.) Apple solved it again but made it faster.
I use sparse disk images extensively as a tool to make certain directories require specific passwords. I simply create an encrypted sparse disk image and set a password on them. Seems like this will speed up my use case.
https://developer.apple.com/documentation/virtualization/vzd...
I guess this is the Apple version of qcow2 and friends
“Asif” is a fun name for a disk image format. It’s as-if it was a real disk. ;)
Andor is a funny name for a tv show
Well, TV show and/or long movie depending on how you look at it.
The benchmarks are weird to me - the ASIF tests were done on M3/4, but for everything else it was done on an M1?
Yeah, that jumped out at me, too.
If they can’t re-run the benchmarks on the same hardware, it’s hard to compare the numbers.
I skimmed through the article, but I have a question that I hope someone can answer. I have a sparse disk image created on a NAS (which runs Linux), and I use it to backup some stuff (not a VM) from the Mac in the native format (the macOS APFS file system).
Would this new format, ASIF, make this faster and better whenever I switch to macOS Tahoe? I hope there wouldn’t be any gotchas with respect to storing this disk image on a NAS.
Yeah I’m interested in this as well. There’s something about the way time machine works over SMB that is absolutely unfashionably dog-slow. I suspect SMB performance on mac is just not very good in general tbf
My kingdom for a documented disk image format.
I'll take your kingdom!
ISO 9660.
https://www.iso.org/iso-9660-images-for-computer-files.html
VMDK
https://en.wikipedia.org/wiki/VMDK
Amiga
https://en.wikipedia.org/wiki/Amiga_Disk_File
UDF
https://en.wikipedia.org/wiki/Universal_Disk_Format
Apple Disk Image
https://en.wikipedia.org/wiki/Apple_Disk_Image
VMDK isn't documented there, and is a container for multiple partially documented - and some only reverse engineered - subformats.
2 replies →
VHDX https://learn.microsoft.com/en-us/openspecs/windows_protocol...
About the only decently documented disk image format I've found is qcow2. [0] Which is usually not the best tool for the job.
So very many of them are just header details + "only works with our tools".
[0] https://www.qemu.org/docs/master/interop/qcow2.html
https://docs.kernel.org/filesystems/ext4/index.html
A filesystem is not a disk image.
5 replies →
What blew my mind recently was that I could store an APFS sparsebundle on a NAS drive, then mount it over NFS and use it as a plain old APFS volume. Despite the filesystem layering, it works pretty much like a local volume, albeit with a bit of performance degradation. Seems preferable to something like iSCSI for using APFS with network storage.
Perhaps this new format would work even better?
Not sure why that would blow your mind, but Apple's old Time Capsule basically mounted a sparsebundle over the network too. And yes I would also guess this new format would work better.
My experience with NFS file management has been less than stellar, so running a full, virtualized, and performant APFS volume on top of it feels like a bit of a magic trick.
2 replies →
My favourite blog specifically for the painting articles
They are amazing, for sure.
Yes! So glad GP brought that up. Bookmarked, looking fwd to spending more time absorbing the fantastic images and accompanying texts.
I wonder with the recent changes regarding containers and now disk images, if Apple plans to enter the server or cloud market.
Would that potentially speed up Docker for Mac and others? (since it's using a vm underneath). That would address a major pain point
They also announced new APIs for native Linux containerization on MacOS with a specific focus on security and performance. This seems like it may be in support of that as well. Anything they can do to improve the performance of containers is a huge win. https://youtu.be/JvQtvbhtXmo?si=3OphClGvylHggmSW
Any chance this will reduce the space needed for major OS updates? Those have always been desperately inefficient.
No. Disk images aren't involved in the OS update process.
They are - in that OS updates are delivered in and extracted from disk images. Not sure how they could be any more involved…
1 reply →
Oh awesome! So it's more virtualisation-focused as opposed to HFS+ --> APFS migration?
Apples and oranges. This is a new disk image format. You can create an APFS _volume_ in the ASIF _image_.
Has anyone found the specification for the format?
I have a meta question not directly related to the article but more about HN itself. I posted this exact link 9h before this submission was posted[1]. How is it possible that there is a new entry for the submission given the link is the same?
[1]https://news.ycombinator.com/from?site=eclecticlight.co
Dupes are allowed after some number of hours.
I think it is time * (votes+comments).
In practice, if it gets any real amount of votes or comments, you have to wait a year to repost. If it doesn't get any attention, it can be reposted quickly (though I think it should be a day later).
It depends. I tried posting a few links I came across over the past few days, and it just showed me dups but they were from days or weeks or sometimes months ago.
2 replies →
I thought that number was significantly more than 9.
It seems like, if the link is the same, submitting a dupe should just raise the original.
My guess is that “fastest gun in the west” might be a bit anti-pattern with respect to community.
Because in ye olden times, mild URL shenanigans seemed to have been a common hack to bypass more strict dupe detection.
And the community probably doesn’t really benefit from aggressive karma seeking —back then being first would give you a point for every resubmission. [1]
But that’s all speculation based on a supposition that what is more likely to be submitted by other users is not a better criterion for choosing to submit.
But I could very well be wrong and probably am.
[1] and number of submissions is probably at best a noisy signal for front page placement and might be negatively correlated with curiosity…I mean even here, what Apple is doing doesn’t stray too far from yesterday’s big press release by one of the most valuable corporations in the world.
Might be something weird with this domain. Look at the list of submissions, yours isn't the first dupe that was accepted in a relatively short time window.
Nice. But I would like it better if the effort was to support ext4, BtrFS, NTFS and other popular filesystems from the Linux and Windows world...
Apple recently made FSKit a supported/documented API, which should allow third-parties to add support for other filesystems: https://developer.apple.com/documentation/fskit?language=obj...
3rd party supporting a file system would be one of the last things on a list of all software I’d ever want a 3rd party writing instead of the OS maker.
Nightmare to evaluate the options, pure stress testing the options, difficult to know if it didn’t mess something up.
22 replies →
Is it actually supported and usable now? I seem to recall it spending a lot of time in a "half-documented and not actually available" state.
Thanks for sharing about this! I didn't know, and I like to use (or at least play with) some third-party filesystems on macOS.
This article describes a new disk image format (on which a filesystem can be put, APFS in the article), not a filesystem, or did I misunderstand?
edit: added the word "image", which I apparently forgot to type. Mentioning the edition because otherwise it would make an answer to this comment difficult to understand.
This is a new disk image format. Not even a disk format. For virtualizing a disk.
On which you can put a filesystem, yes.
There seems to be an extraordinary amount of confusion in many of the comments, I don't know why.
6 replies →
Yeah and OP was asking, why was effort spent on a disk image format rather than a file system. Seems like a reasonable question to ask.
3 replies →
ext4 and Btrfs are only well supported on Linux; they are not universal standards.
NTFS was only supported well on Windows until recently; but extensions like NTFS Encryption (BitLocker) are still Windows only. Mac still does not let you write to an NTFS volume.
APFS and HFS+ are obviously Apple file systems.
FreeBSD does not support ext4 or Btrfs well; but instead prefers UFS2 or ZFS despite also being an open-source Unix-inspired OS.
The world runs on proprietary or non-universal file systems with CDFS (ISO 9660), FAT, and exFAT being the sole exceptions.
Is there a single filesystem in the world (besides "simple" ones like FAT) that both has an open standard AND is licensed in a "usable" codebase (MIT or other "non-copyleft" license)?
17 replies →
> NTFS was only supported well on Windows until recently
I remember using ntfs-3g without any issues on my first Linux laptop 15 years ago. So that's not really "recently".
2 replies →
if Mac also supported Linux file formats they would get closer to universal, though? we can't get to that world without these individual steps.
creating a new non universal one is backing away from it
7 replies →
[flagged]
1 reply →
Does anybody else think that it would make sense for Apple and Microsoft to just get in a room and horse trade a few things like this, if they cared about user experience? Cross-license both APFS and NTFS, and share any internal documentation under NDA so that external drives can use modern formats with safety features like journaling without locking users in.
Oh wait, I just answered my question.
I suspect there wouldn't be an agreement on a minimum set of features for a modern filesystem, even just for external disks, even if you limited it to flash storage devices to avoid all the complexities of spinning platter latency.
After all, there's no such agreement on Linux either - we just have all the Linux vendor options available.
1 reply →
This is a disk image format so why not VHD? It seems like it's open enough and supports what a virtual disk needs what do we gain with yet another file that's a disk type?
Can't abstract away your property that way.
[flagged]
I think this is actually a human struggle - I add Conclusions to my blog posts / dev guides because it feels awkward to just end with the last step.
But it also always feels like I am just awkwardly restating things I just said. It's not a thesis, an article generally not complicated enough to need a re-synthesizing of main points at the end.
I have an unusual way of communicating sometimes that used to be quirky and fun. Now I simply get accused of being a bot. Before LLMs I never got accused of being a robot.
I think the idea of good writing is soon going to morph with writing that doesn’t feel like it was made with AI.
3 replies →
I used to love em dashes. But now I can't use them anymore — people will think it's written by AI.
1 reply →
If you're going to be this superstitious, then you might as well lean into it and have a little fun. Better throw some herbs into the air and do a little jig for good measure. Personally, I'm going to accept that I won't be able to tell, because models are trained and prompted to appear human.
It's a shame that every new cool product/dataformat/cable/cpu/whatever researched by Apple has very little (or no) public documentation. Sure, there are lots of hackers who can test and reverse engineer those pretty quickly, but it's just unnecessary work. I don't know why Apple is so revered in hacker circles, to be honest. Not even Microsoft does this shit anymore, they're open sourcing a lot of research this decade, but they're still seen with extreme distrust. Whereas Apple was always secretive and used underhanded tactics, but it is still loved.
Here's the link to the Darwin, the underlying Unix layer of macOS, which is open source [1].
Go for it!
[1]: https://github.com/apple-oss-distributions/distribution-macO...
You're joking, right? You can see at a glance that this mess is only published for compliance reasons. There is no documentation at all, and most of the code consists of numerous versions of open-source libraries that have been subtly modified due to their licensing requirements, which necessitate disclosure of modifications. Good luck building any flavor of that! See: https://news.ycombinator.com/item?id=35197308
Was this not already a solved problem? Why reinvent the wheel other than for vendor lockin?
It's a solved problem in the sense that Apple solved it more than a decade ago. (The sparsebundle format was introduced in Mac OS X Leopard from 2007. The sparseimage format was even older; I saw discussions of using it with Panther.) Apple solved it again but made it faster.
Doesn’t sound particularly useful, if you aren’t setting up containers.
I use sparse disk images extensively as a tool to make certain directories require specific passwords. I simply create an encrypted sparse disk image and set a password on them. Seems like this will speed up my use case.
Unless they can migrate Time Machine to it. The performance improvements sound like they might be a real boon there.
If Apple would just ship bind mounts and FS/pid namespaces... or even just un-break chroot...
I hope they did something to not load my hard drive with all the AI crap and steal the storage.
This ridiculous arms race can't be over soon enough.