Comment by jitl
2 months ago
bummer this expects NFS; this is all a brand new implementation vs what Cloudflare runs internally in production for DOs? I can’t imagine all your workerds share a single global NFS… What would a more realistic workerd with DO support look like?
ah well, we have celld!
Honestly, you wouldn't want to run our internal implementation -- it's far too complicated. Unless you have 100+ datacenters around the world -- then maybe.
The goal of this new design is to scale to a cluster while being operationally very easy to set up. Ideal for self-hosting.
NFSv4 is an easy first step, convenient because it's broadly understood, has many implementations, and requires no client libraries. I could imagine a follow-up to support LiteFS instead of NFS could make a lot of sense, though it'll get more complicated.
But yes, celld is definitely ahead of us here. No doubt about that.
This just makes me curious, what does the internal implementation actually look like?
I've always wondered what it is Cloudflare does to make DO's work but this specific thing doesn't seem explained anywhere
ZeroFS might help here to put the actual storage on an S3 compatible object store? https://github.com/Barre/zerofs
The ZeroFS readme says
but the commercial license link https://www.zerofs.net/licensing is a 404 page
my interest in s3 is for how easy it is - just need HTTP. mounting a filesystem and worrying about filesystem level locks is more complicated.