I slightly modified the original image node by adding the image ID in the database and replacing the output in the DOM so that the caption under the image is visible.
After that, I noticed a strange behavior when pasting images via the clipboard:
When pasting text containing images, images in webp format are not inserted (other formats are inserted correctly), but replaced with a paragraph. However, if you insert a separate image into the editor, it will be inserted correctly.
After some research, I realized that the problem is certainly not in the image format. But rather in the fact that the layout of the site from which I made cat and paste had images in paragraphs and for some reason this construction throws the image out of the document when copying.
I think the problem is that in my scheme the images should be blocks. And when the parser sees them inside a paragraph, it simply throws\out of it.
But I would like the paragraph to be split into two sections in this case.
I can reproduce this now. The existing parser code assumes that nodes whose ProseMirror representation cannot be placed inside the representation of one of their parent nodes (in the DOM) must be discarded. So if your clipboard data includes the paragraph, which cannot be a parent of your image node, it would drop it.
Attached patch relaxes this requirements. I’m a bit worried about the kind of effects it will have on messy parses, but I agree that dropping the image in this situation is not ideal.
This unfortunately broke our JATS XML parser and, so far we’ve been unable to update to prosemirror-model to 1.25.0+. JATS XML is not as strict about schema as prosemirror and we can’t accept all the content from it automatically. It was very important that parser dropped all the content that was invalid. This also allowed for incremental development: we implemented support for more content cases and node relations gradually while keeping our doc clean and functional.
Would it be possible to add an option to make parser behave as it used to - by passing a config or in any other way that you see fit please?
Thank you!
JATS XML allows for varied and mixed content which, in practice, is extremely hard to fully cover with parse rules. It would also require the supplier of the JATS (essentially a third-party service) to effectively communicate any schema changes to our team, which realistically can’t happen. More importantly, given the high variety of these JATS files - which are customer-provided content - we can’t be confident that our parse rules cover every case that will appear in production. We have very good test coverage for import and export, but none of those tests can guarantee correct import of arbitrary JATS in the wild with the parsing rules relaxed as in 1.25.0+
As a result, deploying our editor to each client - each of whom may have peculiarities in their JATS due to the nature of their content - risks producing a wave of urgent bugs and a reputation for shipping a raw product. This is exactly why we haven’t been able to update to the version where relaxed parsing was introduced, or to any version since.
Relaxed parsing is a great feature in many cases, but it shouldn’t be the only option. We’d like the ability to disable it in code and opt into strict parsing, so that content which doesn’t match the schema is dropped rather than accommodated. I propose to add a NodeSpec property to flag a node for strict parsing or pass the flag into Schema constructor - new Schema<Nodes, Marks>({…, parseStrict: true })
ProseMirror’s DOM parser has always been a messy, heuristic thing that tries to use as much of the document content as possible, and discards the rest. That’s not a new thing, and it’s not going to change. It sounds like for your use case you might want to implement a pre-processing pass or a custom parser.
That is true as with the most parsers perhaps. However changing the way parser works in such fundamental way is significantly disruptive, as you, actually, worried yourself in your message above. And to create our own parser just to achieve the same behaviour we have in 1.24.x doesn’t seem rational.
I have to say that even on the philosophical level relaxed parsing is a bit contraditctive to the nature of prosemirror that is built around strict schema, which makes prosemirror to stand out that much and allows to implement complex editing in a controlled and predictable way. Loose parsing into strict schema will always be prone to flakiness. If anything it is the relaxed parsing that should’ve been optional.
I’m sorry to bring that topic back that much later after it was originally discussed. We’ve actually attempted to upgrade our dependencies about 8 months ago, but, having ran into this issue, postponed to follow further development of the library.
Do you have any concerns with making strict parsing optional? Would you be open for me creating a PR for this to discuss?
Not really. The output of the parse will strictly conform to the schema. The input is treated as random HTML.
The library does not contain an implementation of strict parsing. What that even means is debatable. You seem to be expecting a specific type of non-strict parsing, where some nodes are discarded. I don’t think formalizing the specific behavior of pre-1.25 parsing is something I want to do.
It will conform indeed. But how will it be produced? Promoting or merging nodes to achieve that conformity just adds to the unpredictability. So yes - a document created is conforming but semantically the nodes created are less likely to be structurally meaningful. In other words the ultimate problem of source incompatibility with the schema is resolved by compromising validity and value of the content that we actually managed to accommodate properly into the schema. Hence the philosophical conflict - the structure of schema is there but, filled with content that perhaps wasn’t supposed to be there, questionable in meaning. After all, if schema says something has to be there and nothing else - so it should be. What to do with the content that doesn’t fit schema - that should be a product level concern and developer should be able to make decisions about that.