Comment by qhwudbebd
2 days ago
AFAIK fly.io run firecracker and cloud-hypervisor VMs. This seems to have a copy-on-write filesystem underneath.
Given their principled take on only trusting full-VM boundaries, I doubt they moved any of the storage stack into the untrusted VM.
So maybe a virtio-block device passing through discard to some underlying CoW storage stack, or maybe virtio-fs if it's running on ch instead of fc? Would be interesting to hear more about the underlying design choices and trade-offs.
Edit: from their website, "Since it's just ext4, you won't run into weird edge cases like you might with NFS or FUSE mounts. You can happily use shared memory files, for example, so you can run SQLite in all its modes." So it's a virtio block device supporting discard that's exposed to the VM. Interesting; fc doesn't support virtio discard passthrough, and support for ch is still in progress...
I have a post coming next week about the guts of this thing, but I'm curious why you think we'd avoid running the storage stack inside the VM. From my perspective that's safer than running it outside the VM.
My impression is that you (very reasonably) treat anything inside the VM as untrusted. If you want trusted rollback, presumably that implies that the VM can't have any ability to tamper with the snapshot?
But maybe you have parts of the stack that don't need to be trusted inside the VM somehow? Looking forward to the article.
Safer from what? It depends whether you're protecting the infra or the data.
They're closely linked; protecting the infra is protecting the data.