Hello ProseMirror enthusiasts
I’m the author of a CRDT library Yjs that handles automatic conflict resolution on shared data. I want to share with you a p2p, offline-capable, shared editing demo using ProseMirror I’ve been working on for a couple of months. Visit our website https://yjs.dev in two browsers and observe how the documents sync.
Here is why this demo is really cool:
After your first visit to the website, it is available offline. I use service workers to make the resources available offline. The Yjs document that holds the state of the ProseMirror document is persisted using y-indexeddb to the local browser database. All content that is created while offline is synced to the other peers when you reconnect to the internet.
This demo uses y-webrtc to share document updates directly with other peers without a central instance to handle conflict resolution. I’m pretty brave to use y-webrtc on a public instance. I do this to test the reliability of the webrtc network. At some point I might switch it out in favor of y-websocket, which is much better suited to handle a large number of visitors.
The content will appear to sync instantly. Browser tabs communicate directly with each other (without WebRTC) using broadcastchannels, completely skipping network communication. This is also the technology that makes it possible to sync content between browser tabs while offline.
You can version the state of the document. When you click on a version, the changes are highlighted by the user who created them. Because this is a public instance without real user management, there is only a “local” and a “remote” user (your browser vs. everybody else). When you come back after a while, click on “Changes since last version” to see what happened while you were gone.
- In order to support thousands of users visiting the website and handle their sync-conflicts, Yjs needs to be able to represent the data very efficiently. There is a short outline here about the data representation techniques I use to make this performant. Compared to other CRDT implementations, Yjs is up to 1000x faster and encodes data 300x smaller than un-optimized CRDTs: https://github.com/dmonad/crdt-benchmarks
- Versions are just views on the data. Normally, structs are transformed to tombstones when they are deleted. The demo transforms un-needed structs to tombstones locally until a version is created. Therefore, the data model does not grow unboundedly for all clients. It will only grow if you create versions.
- The webrtc connector creates a totally connected mesh network of webrtc connections. It’s not a problem if some of the connections fail, as long as there is a path from every client to every other client (the graph is connected). After a threshold of about ~30 clients, the y-webrtc provider intentionally creates a partially connected mesh network. There is a good chance that all data will still sync between all the clients. But there are no guarantees, therefore I highly recommend other communication protocols when a large number of clients is expected.
- https://github.com/yjs/yjs-demos has code examples for Yjs + ProseMirror / Prosemirror+versioning and for a third-party editor that is built on ProseMirror (Atlaskit).
- ProseMirror + CRDT's? Initial thread on CRDTs in ProseMirror that roughly describes how the y-prosemirror binding works.
- https://demos.yjs.dev/prosemirror/prosemirror.html is a simple ProseMirror demo using y-websocket, without versions and offline editing. Use it to inspect network traffic.