Comment by josephg

5 years ago

Author here. I love this comment.

> So, having said all of that, is there hope for adding things like this to my toolbelt?

Yes; but not yet. Right now if you want you can use yjs or sharedb. But the APIs aren't spectacular. Eventually I want these things integrated into svelte / react / swiftui and postgres / sqlite / whatever so it just sort of all works together and we get nice tutorials taking you through what you need to know.

We aren't there yet. Automerge is slow (but this is being worked on). Yjs is hard to use well with databases, and its missing some features. We'll get there. The point of all the mathematical formalisms is that if we do it right, it should just work and you shouldn't have to understand the internals to use it.

9 comments

josephg

Reply
kevinjahns  5 years ago

What are you missing from Yjs?

I'm sharing the vision with you that things should just work. It literally takes you two lines of code to make any of the supported editors collaborative. The Yjs types are a minimal abstraction over the crdt model. I hope that other people build more abstractions around Yjs (for example a react store that simply syncs).

The thing is.. all of what you are talking about is already possible. A wasm port will eventually bring some performance improvements. But Yjs is already very close to optimal performance and it works in all browsers.

But I really am interested in more feature request. I'm mainly interested in building collaborative apps with CRDTs. CRDTs as a data model is an entirely new concept and we need to get experience on how to build apps with CRDTs. And also we need to find out what is still missing.

So head over to discuss.yjs.dev and share your experience.

  • josephg  5 years ago

    I want yjs to have:

    - Support for moving objects (as well as inserting and deleting)

    - Better support for long lived documents. (I want to be able to pull historical deleted stuff out of a document so it doesn't grow unbounded).

    - Presence information and markers (this is sort of supported now?)

    - The gorgeous automerge API frontend ported across

    - Support for interacting with yjs from rust / C / swift / etc

    • kevinjahns  5 years ago

      - The gorgeous automerge API frontend ported across

      I want all of that too. Although I would add Automerge's API on-top of Yjs as an additional layer. It really does play nicely with Web Frameworks like react. Although for building editor bindings I would choose the Yjs types because they allow faster access to the data. Yjs only provides a minimal abstraction around the CRDT. I hope that other people will build nice stores/abstractions around Yjs that make interacting with shared data easier.

      - Presence information and markers (this is sort of supported now?)

      Presence is handled by a different CRDT (the Awareness object in y-protocols). All providers implement the Awareness protocol. In many cases you want one awareness instance for several documents, so it makes sense to separate them.

      Markers is probably what I refer to as relative positions: references to specific positions in the document. It even plays nicely with undo-redo.

      - Better support for long lived documents. (I want to be able to pull historical deleted stuff out of a document so it doesn't grow unbounded).

      Yjs does this reasonably well. To a point that you can even build 3d applications like relm.us with Yjs. But you are right that more work could be invested in researching even better solutions.

      - Support for moving objects (as well as inserting and deleting)

      Moving of elements can already be modeled with Yjs [1]. Although the approaches described are not very convenient to use. Automerge implements this feature natively, but I'm a bit hesitant to implement the next best approach because I see many downsides that conflicts with features I plan to implement. There is a discussion thread where I outline a few of the tradeoffs [1].

      [1]: https://discuss.yjs.dev/t/moving-elements-in-lists/92

jezclaremurugan  5 years ago

Loved your blog - but I think the parent's question was how to gain the skills of working/designing OT/CRDTs etc on their own. (I am clarifying the question - since I'm interested in the answer as well)

  • vanderZwan  5 years ago

    I honestly think a good skill to have as a programmer is to know when something is a complex, niche problem with inherently "fragile" implementations (as in, it is really easy to make mistakes) that is therefore best solved by a few domain experts working on an open source library, and just use that library.

    Based on everything I've seen about them, OT/CRDTs sound like they are that.

    • josephg  5 years ago

      (Author here) I agree with that conclusion. I think they're really fun to implement and explore. I've been working in the area for a decade. And I've implemented over a dozen variants of OT systems by now. And despite all that I still consistently make mistakes that are only found with a fuzzer. Even thorough unit testing is never quite enough.

      I recommend implementing them for the fun of it. I love it. But don't put anything you write into production until you can leave a randomized test suite running overnight without errors.

      Example:

      Random op generator: https://github.com/ottypes/text-unicode/blob/master/test/gen...

      Which powers this fuzzer: https://github.com/ottypes/fuzzer/blob/master/lib/index.js

    • UK-Al05  5 years ago

      This guy just asked how to get into them because he is interested in them.

      You just replied don't. Have you considered he wants to this regardless of not applying them purely commercially and wants to perform research, or in a novel industrial capacity?

      1 reply →