ProseMirror + CRDT's?


#1

I recently checked out the automerge library, which uses an implementation of conflict-free replicated data types (CRDT’s) to allow collaborative modifications to a JSON-like state, in a peer-to-peer friendly way. It comes with a WebRTC reference implementation for collaboratively editing documents p2p.

I’ve read Marijn’s blog post contextualizing why ProseMirror uses Operational Transformation fo facilitate collab, but am generally curious: could ProseMirror be made to work with CRDT’s? Automerge itself not be a helpful reference point (it’s not clear how something like ProseMirror’s schema rules would be enforced), but a peer-to-peer compatible layer for ProseMirror seems like a cool and compelling use case.

I’d love to get a sense of what challenges might get in the way of making ProseMirror CRDT compatible; I’m imagining it’s a difficult problem (consensus! convergence!), but could open up some interesting uses for ProseMirror…


#2

No one has worked on this yet, so there’s not much I can tell you. But research in that direction would be interesting.


#3

OK I spent some time playing around with a really naive concept of what this might look like, and threw the results into a repo here:

Automerge is able to model any JSON-like object, tracking any changes made to that object as a series of operations. The code in prosemirror-automerge:

  • converts the Node.toJSON() representation of a ProseMirror doc into an Automerge doc
  • provides a ProseMirror plugin which takes incoming Steps from a transaction, and attempts to “apply” them to the Automerge doc
  • takes incoming changes from a remote Automerge doc, and attempts to translate them to ProseMirror steps suitable for application to the current editor’s doc

There’s a demo in the repo where two EditorView's are wired together with a basic Automerge syncing implementation, and it partially works for basic text-insertion-y types of transforms. It doesn’t work at all for deletions/marks/basically “everything else”.

Some extremely loose thoughts based on this experience:

  • it would be handy to have a way to project the indexing scheme used by ProseMirror onto a plain object outputted by Node.toJSON(). I used the ResolvedPos.path private variable to implement this, but I’m 100% certain this would break and isn’t the correct way to do this:

… in my mind this would look something like the signature of the Node.descendants method, except instead of yielding ProseMirror Nodes, it would just be plain objects (plus the current position). probably this isn’t too hard to implement, my brain is just sputtering on a clean way to do so :slight_smile:

  • in a similar vein, I wanted some way of taking an incoming Step and trying to figure out a minimal representation of the delta change in data as it pertained to the transaction’s tr.doc.toJSON(). Almost like something you could describe as a JSON patch.
  • in general, representing ProseMirror docs as Automerge docs works well, provided you can efficiently translate between the two systems; one finicky aspect of this is that for Automerge to detect incremental changes, I needed to represent ProseMirror text nodes’ .text property as an array of characters (rather than a string) so that changes to individual characters would be seen as having originated from distinct “actors” (in Automerge parlance)

I’m not totally sure how productive it is to continue with this experiment, as I have a hunch (unproven!) that there are fundamental issues which would make Automerge’s data modeling incompatible as-is with ProseMirror.

I am, however, curious to understand the fundamental merging strategies that are described in this academic paper. Automerge decorates its data structures with an array of ops, which look suspiciously like a ProseMirror Step (insertion/deletion as position x/y/z, with contents abc). My somewhat hazy understanding of CRDT’s in general is that they’re not dissimilar from ProseMirror’s indexing system, with the addition that each position has a unique id which corresponds to the person responsible for having created the position. So a document like:

0   1 2 3 4    5
 <p> O n e </p>

Might be represented as:

0{user:1}   1{user:2} 2{user:1} 3{user:1} 4{user:1}    5{user:1}
        <p> O         n         e                 </p>

I’m sure there’s an enormous amount of complexity that I’m eliding here, but I would be curious to understanding if something like this would be possible to implement directly with ProseMirror…