Comment by monkeyelite
9 days ago
There is no reason for every application to have its own sync platform. I suspect this framing came out of mobile apps where there is no composability or modularity between programs.
If you really embrace "local first" just use the file system, and the user can choose from many solutions like git, box, etc.
I hate signing up for your sync just as much as any other SAAS, but it's even more opaque and likely to break.
I agree that not every app needs it's own sync engine, but I disagree with your framing that the file system is the universal way to embrace local first. I have two reasons.
First is that yeah, local first, but I also want concurrency. If it's just local first, you're right, any old sync will do. But I want more than that. I want to not have to think (a la dropbox, being slick). I want my wife and I to be able to make separate edits on our phones when we're in a dead zone.
Second is that sync works a lot better when it has deep knowledge of the data structure and semantics. Git and box both have significant shortcomings, but both exacerbated by the concurrency desire.
But this problem isn't going to be solved by every app making its own sync system. Even if there is a magic library you can adopt that does pretty good, then everyone having their own completely independent hosting solution and sync schedule.
If files are insufficient, what data-structure would make modular sync possible for multiple applications in an OS?
And I’m not suggesting one doesn’t exist, I’m challenging to present a comprehensive solution, that probably involved operating systems.
> I want my wife and I to be able to make separate edits on our phones when we're in a dead zone.
Files do this.
I agree that files solve some rudimentary cases, but they do not even allow simple conflict resolution. Eg. compressed files, including container formats like OpenOffice (text files in a ZIP archive IIRC), might be simple to apply changes from two sides if they are in distant parts, but syncing full files simply barfs.
Note that this does not even need two users: I hit this problem with a desktop and laptop and self-hosted NextCloud myself.
In general, a filesystem that actually stored both raw data (to fail-over to), but also a per-format event log, and maybe even app specific events (imagine a PNG changes, we could have any change recorded as raw bytes, generic bitmap image operation like "modify pixels at x,y to ..." and app-specific log like "gimp: apply sharpen filter on polygon area ...").
This would allow the other side to attempt to do the smartest sync it has (if it has a compatible version of gimp, it could decide to apply the filter, otherwise fall back to raw pixel changes if no conflicts, and then fall back to full file contents reconciliation).
Just like MIME handlers get registered, if file systems provided such change logs, some could have very advanced sync systems with this support from "filesystems".
2 replies →
Files come with certain restrictions, which don't matter for certain types of applications. But for others they do.
I think it boils down to provenance and concurrency. If we edit the same line a file, that's ba merge conflict when it really should be simple and something I shouldn't have to bother with. And when we do do the same line edit, I'd love to have provenance on that data.
Granted, those aren't local first thing exactly, but I think there will be apps that want all of that.
If the app is designed for it you can use a hybrid approach, where a given "document" is stored in 1 file for each client, and the client merges the changes across all files. That way there's never a change conflict that something like Dropbox needs to handle and it can all be offloaded to the app.
I mostly agree with this, but sometimes it's not that simple in practice. I created an app that did exactly this and it resulted in inevitable file conflicts because I couldn't negotiate between the clients when a file should be allowed for editing.