I wrote a mergeStepsArray that receives an array of steps and returns an array of steps with as much merging as possible:
function mergeStepsArray(steps) {
let res = [], current = steps[0];
for (let i = 1; i < steps.length; i++) {
let newStep=current.merge(steps[i]);
if (newStep) {
current = newStep;
} else {
res.push(current);
current = steps[i];
}
}
res.push(current)
return res;
}
(as a side note, writing this was tricky and even though it seems to work fine I don’t know if it’s totally correct or efficient and I think it would be good for the framework to provide a built in way to do it)
The problem is if I run this on the steps returned from collab sendableSteps and store the result on the my server, collab is now out of sync with the server - it has more steps ! so this is not workable.
Can collab be extended to support merging ? this is quite essential as otherwise we get a huge amount of transactions stored on the db (one for each letter typed etc). If there’s only one client typing, hundreds of steps can be combined into one.
or is there another way to solve this? perhaps restart collab from the correct version after each send ? (ugly)
A) This isn’t that much data, in most situations, and B) there’s no reason to store the same data that you sent as part of the collaborative editing in your DB for long-term storage.
My goal is to try to make PM work with a nosql db and no server side (ie Firebase etc). I think this can be in general a very useful application. The problem is the nature of such a db is that transactions operate on a single key, so adding multiple steps in a single transactions is problematic. I will need to do a transaction on the parent - the one holding all steps and that is extremely inefficient as it means sending all versions back to the client.
So instead of that I simply store the result of sendableSteps which has version as key and a dictionary of zero based index:step as value. I then listen for new children starting at root\version == getVersion. This works, but the down side is that the versions on the db are not sequential: version 0 with steps 0-4 then version 5 with steps 0-2 etc which is ugly and means I have to ‘flatten’ the server response to an array of steps on the client.
The bigger problem is with merging steps as explained above. Basically the underlying issue is that collab assumes a flat steps structure so that version is the step index which is not the way I want it in my case.
Another problem is that implementing snapshots get tricky, as I need to store what step index does the snapshot version corresponds to.
(as a side note there is a slight advantage to this structure, I can store clientID only once per version instead of with every step).
What I need is a way to manage collab versions myself. Basically I am asking for a reset(state, version) method in collab that will reset the plugin with an editor state and a version and an empty local ‘history’ (empty steps array). or even better, for receiveTransaction to accept an optional version argument and reset the internal version to that argument. This will allow me to do the version bookkeeping myself and solve the above issues in a clean way.
Another approach, IMO the best but require an API change, is to remove version handling from collab altogether. That is remove getVersion and remove version from plugin initialization and the return of sendableSteps. To achieve the current behavior, the client will simply hold it’s own version variable and do version+=steps.length after calling receiveTransaction and use that version instead of the one returned from sendableSteps when needed.
It sounds like you want to rearchitect the way collaborative history works. Feel free to try – it might well be possible to write a new implementation without touching any of the other modules – but no, I’m not going to rethink that module (which is the way it is for good reasons) just to fit your use case.
I did manage to make it work in the end merging the steps just when sending while keeping them unmerged on the collab of the client who issued them seems to work fine.
I still think that having versions numbers bookkeeping internal to the collab plugin is very confusing (sure confused the hell out of me!). It makes you think that collab plugin actually uses the version internally (for rebasing etc) where in fact all it does is increase the version on recieveTransaction and return it in getVersion and sendableSteps. I really don’t see the point in doing that … it is much better (and trivial) for the client code that uses the plugin to handle the version bookeeping which allows it to do it differently (as in my case - a single version for each send). So again I suggest removing getVersion() and the version argument to init, receiveTransaction and sendableStep.
I also just now wanted to open a thread about this, but found this existing one.
I thought that this is relatively easy to add. We could add meta information to the step about how many versions it consist of. And when merging, collab module could use that information instead of just adding 1 for each step to increase the local version. So local state of the client would consist of multiple small steps (to support undo/redo) but when sending those steps out to the server, we could merge them into a large step and setting meta information. Other clients would get it, apply that larger step and increase version. And the client would get it back as well, see that clientID matches that client, and confirm the local steps based on that meta value as well.
I think this could be done in a relatively simple addition. Maybe collab module could just add this concept of meta information about the number of versions in a step, and then if users of ProseMirror want to merge anything, they could do that and manage meta information properly.
merging the steps just when sending while keeping them unmerged on the collab of the client who issued them seems to work fine.