Wired failure on step.apply(doc) on same set of steps

1

Hi. I have encounter a wired behaviour that I have an array of steps in JSON format being saved in the database. For persisting the steps I have a validation check that will load all steps from the database and call step.apply for all past steps and the incoming steps.

For unknown reason, I encounter that for the same set of steps, the validation passed without error. But later on will have exception throw when calling step.apply.

Do you have any idea to this wired situation?

2

Nope, that shouldn’t happen (applying steps is a pure operation, and applying the same step to the same document should give the same result). You may have some data corruption somewhere—for example accidental mutation of an object or array that is shared between data structures because it is supposed to be immutable.

If you can reduce the issue to a minimal example, that might help.

3

Ya. I think it’s not possible too. Will try to see I can record one example if it happens again.

4

I found out the reason. Seem there is a mergeSteps function being run when saving to the mongo. And the merged steps are invalid when replay. After removing that, the problem is solved.

btw, if we really want to merge the steps before saving into mongo in order to reduce the number of step. Any suggestion on how to do that?

input
{
    "payload": [
        {
            "stepType": "replace",
            "from": 69,
            "to": 70
        },
        {
            "stepType": "replace",
            "from": 69,
            "to": 70
        },
        {
            "stepType": "replace",
            "from": 68,
            "to": 71,
            "slice": {
                "content": [
                    {
                        "type": "text",
                        "text": " "
                    }
                ]
            }
        },
        {
            "stepType": "replace",
            "from": 69,
            "to": 71,
            "structure": true
        },
        {
            "stepType": "replace",
            "from": 69,
            "to": 71,
            "structure": true
        }
    ]
}

merged
{
    "payload": [
        {
            "stepType": "replace",
            "from": 69,
            "to": 71
        },
        {
            "stepType": "replace",
            "from": 68,
            "to": 71,
            "slice": {
                "content": [
                    {
                        "type": "text",
                        "text": " "
                    }
                ]
            }
        },
        {
            "stepType": "replace",
            "from": 69,
            "to": 73,
            "structure": true
        }
    ]
}

Is that because of the structure?

5

Merging steps should generally be safe, if you can make sure that the document versions between the merged steps aren’t going to be used (which can be tricky to do with collaborative editing). If you find a sequence of steps that is valid individually, but can’t be applied when merged, let me know, since that would be a bug.

6

Let me explain the flow that we’re having right now. Not sure it contribute the the bug or not.

  1. Web will emit steps with collab.senable.
  2. Server will merge those steps with the following function for persistence storage.

Created a repo to reproduce the bug

7

Thanks for the test case. It did indeed have to do with the structure flag—you hit a rare case where two structure-changing steps merged into a step that couldn’t actually be applied anymore. This patch should prevent such merging from happening (released as prosemirror-transform 1.2.12).

8

Thanks for the fix.

9

btw, would like to know the correctness of the mergeSteps function. Is this the correct way to merge steps?

10

Yes, the function looks good.

1 Like