Comment by commandersaki
6 months ago
Video about it here: https://developer.apple.com/videos/play/wwdc2025/346/
Looks like each container gets its own lightweight Linux VM.
Can take it for a spin by downloading the container tool from here: https://github.com/apple/container/releases (needs macOS 26)
The submission is about https://news.ycombinator.com/item?id=44229239)
The former is the framework enabling Linux containers on lightweight VMs and the latter is a tool using that framework.
> Looks like each container gets its own lightweight Linux VM.
That sounds pretty heavyweight. A project with 12 containers will run 12 kernels instead of 1?
Curious to see metrics on this approach.
This is the approach used by Kata Containers/Firecracker. It's not much heavier than the shared kernel approach, but has significantly better security. An bug in the container runtime doesn't immediately break the separation between containers.
The performance overhead of the VM is minimal, the main tradeoffs is container startup time.
I wonder why Apple cared so much about the security aspect to take the isolated VM approach versus shared VM approach. Seems unlikely that Apple hardware is going to be used to host containerized applications in production where this would be more of a concern. On the other hand, it's more likely to be used for development purposes where the memory overhead could be a bigger concern.
2 replies →
Ram overhead can be nontrivial. Each kernel has its own page cache.
2 replies →
Is that not the premise of docker?
No it's the opposite, the entire premise of Docker over VMs is that you run one instance of all the OS stuff that's shared so it takes less resources than a VM and the portable images are smaller because they don't contain the OS image.
3 replies →
Nope, docker uses the host's kernel, so there are zero additional kernels.
On non-Linux, you obviously need an additional kernel running (the Linux kernel). In this case, there are N additional kernels running.
10 replies →
no.
I could imagine one Linux kernel running in a VM (on top of MacOS) and then containers inside that host OS. So 1 base instance (MacOS), 1 hypervisor (Linux L0), 12 containers (using that L0 kernel).
That's how Docker Desktop for Mac works. With Apples approach you have 12 VMs with 12 Linux kernels.
Also works on macOS 15, but they mentioned that some networking features will be limited.
Shoutout to Michael Crosby, the person in this video, who was instrumental in getting Open Containers (https://opencontainers.org) to v1.0. He was a steady and calm force through a very rocky process.
"A new report from Protocol today details that Apple has gone on a cloud computing hiring spree over the last few months... Michael Crosby, one of a handful of ex-Docker engineers to join Apple this year. Michael is who we can thank for containers as they exist today. He was the powerhouse engineer behind all of it, said a former colleague who asked to remain anonymous."
https://9to5mac.com/2020/05/11/apple-cloud-computing/
We can thank the linux kernel developers for implementing namespaces and overlayfs.
And we can thank predecessor systems like BSD jails, Solaris zones, as well as Virtuozzo/openVZ and lxc as previous container systems on linux.
Docker's main improvements over lxc, as I understand it, were adding a layered, immutable image format (vs. repurposing existing VM image formats) and a "free" public image repository.
But the userspace implementation isn't exactly rocket science, which is why we periodically see HN posts of tiny systems that can run docker images.
I would assume that "lightweight" in this case means that they share a single Linux kernel. Or that there is an emulation layer that maps the Linux Kernel API to macOS. In any case, I don't think that they are running a Linux kernel per container.
You don’t have to assume, the docs in the repo tell you that it does run a Linux kernel in each VM. It’s one container per VM.
Good call, thanks for clarifying!
"Lightweight" in the sense that the VM contains one static executable that runs the container, and not a full fledged Ubuntu VM (e.g. Colima).
It seems to work on macOS 15 as well, with some limitations[0].
[0] https://github.com/apple/container/blob/main/docs/technical-...
interesting choice - doesn't that then mean that container to container integration is going to be harder and a lot of overhead per-container? I would have thought a shared VM made more sense. I wonder what attracted them to this.
It seems great from a security perspective, and a little bit nice from a networking perspective.
The "one IP per container" approach (instead of shared IPs) is similar to how kubernetes pods work.
I can see the decision to do it this way being related to their private secure cloud infrastructure for AI tools.
I like the security aspect. Maybe DNS works, and you can use that for communication between containers?
> Looks like each container gets its own lightweight Linux VM.
We're through the looking glass here, people
"Containers" now apparently means "boot a docker image as an ephemeral VM."
Which isn't such a bad idea really.
"Looks like each container gets its own lightweight Linux VM."
Not a container "as such" then.
How hard is it to emulate linux system calls?
> How hard is it to emulate linux system calls?
It’s doable but a lot more effort. Microsoft did it with WSL1 and abandoned it with WSL2.
Note that they didn't "do it" for WSL1, they started doing it, realized it is far too much work to cover eveything, and abandoned the approach in favor of VMs. It's not like WSL1 was a fully functioning Linux emulator on top of Windows, it was still very far from it, even though it could do many common tasks.
19 replies →
> How hard is it to emulate linux system calls?
FreeBSD has linuxulator and illumos comes with lx-zones that allow running some native linux binaries inside a "container". No idea why Apple didn't go for similar option.
FreeBSD Linux emulation is being developed for 20 (may be even 30) years. While Apple can throw some $$$ to get it implemented in a couple years using virtualisation requires much less development time (so it’s cheaper).
Apple's already got the Virtualization framework and hypervisor already (https://developer.apple.com/documentation/virtualization), so adding the rest of the container ecosystem seems like a natural next step.
It puts them on par with Windows that has container support with a free option, plus I imagine it's a good way to pressure test swift as a language to make sure it really can be the systems programming language they are betting that it can and will be.
OrbStack has a great UX and experience, so I imagine this will eat into Docker Desktop on Mac more than OrbStack.
Because that‘s a huge investment for something they have no reason or desire to productivize.
syscalls are just a fraction of the surface area. There are many files in many different vfs you need to implement, things like selinux and ebpf, iouring, etc. It's also a constantly shifting target. The VM API is much simpler, relatively stable, and already implemented.
Emulating Linux only makes sense on devices with constrained resources.
> How hard is it to emulate linux system calls?
Just replace the XNU kernel with Linux already.