Comment by aitchnyu
2 days ago
Max, can users be limited to delete/push/force push certain branches? Is a server repo incremental backup-friendly? Is is as efficient in storage space and data transfer as regular git? Can we incrementally sync dbs like Sqlite?
Seems this is the best E2EE storage. No choosing creator's cloud vs finicky self hosting, nor worry about flaky sync solutions for my password manager, note app, photo storage etc. A 2$ vps would be overkill.
Thanks for these great questions!
- limiting users to delete/push/force; this is possible but I don't see how to cryptographically guarantee it. The server can't really help since it doesn't know what's a pack, index data block or ref. The clients can enforce this policy, but then it would be possible to make an evil client that skirts the policy. How much protection do you think you need?
- the server repo right now is implemented 100% as a postgresql DB, so yes, I think that means it's incremental backup-friendly? [1]
- e2ee git has trouble being as efficient as regular git since the server can't tell you which blocks it has; however, there are pretty good optimizations made using indices and packfiles, the white paper has more details, and I hope to write a blog on it soon.
- I'm not sure about the sqlite question. Is there a good way to backup sqlite incrementally over standard git? If not, then maybe the KV-store is better fit for this application.
I agree that git over E2EE is the best storage, even for things like PDFs and photos. Yeah, FOKS should be hostable with a very thin VPS. The storage needs will scale (n log n) as the number of users due to the Merkle Tree, but for small n, this is likely fine!
[1] https://www.postgresql.org/docs/17/continuous-archiving.html...
Can we fetch incremental changes from KV-store? I want to read the KV store docs. Could you share that?
Why is server repo specifically Postgres?
Thanks for the answers!
No incremental fetch right now other than what postgresql provides by default. If you're hosting a FOKS server, there is important metadata to backup too.
The best docs for the KV store are in the white paper, Section 5.1. White paper is linked to from foks.pub
Postgres seemed to check all the boxes for me. I wanted to keep things very simple at first in terms of setup and ongoing management, so didn't introduce other storage backends for other parts of the system.