Comment by blixt
6 months ago
I agree. In theory you need to just distribute serialized patches but in a real world backend scenario you may need to integrate with knowledge of current document state, user identity, and possibly even some level of access control.
I’ve wanted to use Y.js with a Go backend multiple times but gave up each time due to time constraints as it’s hard to find simple reference implementations.
I’ve been checking back over the years but it still seems hard to do this outside of Node.js.
Same experience. Go is my default choice for backend too
It can easily be a full time job
You need to mirror all logic and encoding/decoding part
https://github.com/yjs/yjs/blob/main/src/utils/encoding.js
https://github.com/yjs/yjs/commits/main/src/utils/encoding.j...
I was thinking about spining off nodejs instance just for persistence and syncing with postgres
Author here — I think our own almost server is almost exactly what you’re looking for? Y-Sweet [1] is open source, written in Rust and persists documents to S3. We offer a hosted version if you want to quickly check it out, which you can use with your own S3 bucket if you want to self host eventually.
This is the first I’ve heard of Platejs, but we do have a tutorial on integrating a block editor using BlockNote [2]
[1] https://github.com/jamsocket/y-sweet
[2] https://docs.jamsocket.com/y-sweet/tutorials/blocknote
Fyi, building a collaborative editor right now and it's still hard to do outside of Node
Still better time spend than doing leetcode.
It's my go-to argument against leetcode style interviews.
I would rather ask a candidate to spend 30 mins and do research together on collaborative editing, or visualizing distances used in pgvector or similar vector database.
Imagine how far the whole colabarative editing space moved forward if 1% of leetcode grinding were rerouted
I'm working on an Obsidian plugin called Relay [0] that makes Obsidian real-time collaborative. Users can share specific subsets of their vault with different groups of people. Our goal is to make Obsidian suitable for corporate use.
We use y-sweet for real-time collaboration backends (thanks paulgb and team!) and handle authentication via Pocketbase. We host everything on fly.io.
I believe there's a lot of interesting space between "pure" local-first software and being a data serf to modern SaaS applications.
On the technical side, it makes sense to have most of your graph be server-managed but locally mirrored. We maintain a store of all user-accessible metadata on the user's device while keeping it up-to-date with SSE – similar to a simplified version of Replicache. Pocketbase makes this very convenient to work with. We modified Pocketbase to use h2c, enabling end-to-end HTTP/2 through fly's edge proxies. This allows us to serve numerous SSE connections for collections related to authentication, authorization, file sync, and other metadata.
We aim to keep all documents on the user's machine (following Obsidian's file-over-app principle). This means the Obsidian client needs to sync with files on disk that lack CRDT context when the codemirror editor is closed. We've developed novel solutions using a combination of diff-match-patch and a diff review tool that we surface to users in these scenarios.
One cool thing about using y-sweet's in your architecture (eg. separate collaboration backends), is that your customers could self-host the collaboration server, preventing you from accessing their data. This business-friendly version of partial-self-hosting (BYOC?) creates nicely aligned incentives for a more ethical kind of SaaS company -- you can build pretty much everything open source except the auth server, and then charge for authn/authz in order to fund development of the project, and the customer gets sovereignty over their data.
One interesting feature on fly.io is the `fly-replay` header, which enables dynamic request routing of websocket connections to machines [1]. This is not a redirect that the user can see, rather this allows the edge proxy to replay the connection to another host transparently to the user. Combined with fly.io's auto start/stop functionality [2], you can build something similar to jamsocket/plane with minimal code and some nice properties like controlling geographic placement and having a single URL for all document servers.
It is a super exciting space to build in right now. I'm grateful for yjs and to the jamsocket team for everything they've built (and open-sourced!).
[0] https://system3.md/relay [1] https://fly.io/docs/networking/dynamic-request-routing/ [2] https://fly.io/docs/launch/autostop-autostart/