RFC: 'Node views' to manage the representation of nodes

Show
1

In response to several questions and discussions about embedding other editing components in a ProseMirror view in order to edit certain types of nodes, as well as the issue of it being unreasonably hard to show placeholders for empty textblock nodes, I finally tried to set up such a thing myself. My main test case was having a CodeMirror instance inside a ProseMirror instance, for editing code blocks.

Turns out it wasn’t quite as easy as I expected, so I’ve been working on a design to make this easier. There are several problems:

  • Applying changes made in the custom node representation to ProseMirror’s state.

  • Fighting between ProseMirror’s selection handling and the selection in nested controls

  • Cleverly updating the nested control instead of redrawing it every time it changes (i.e. #378)

You can kind of hack around these with the current API, but the code will be awful, inefficient, and fragile.

My proposal for a solution to this is a feature called “node views”, which are little views in the same spirit as the editor view, i.e. objects that manage a piece of the DOM, keeping as little internal state as possible, and simply displaying values that they are given.

In the node-view branch of the prosemirror-view repository, there’s a commit for which I’ve pasted the added doc comments below:

EditorProps.nodeViews: ?Object<(node: Node, sendAction: (state: EditorState, pos: number) → ?Action) → NodeView>

Use the given node view implementation to manage the DOM representation of certain node types. When a node view is created, it is passed an action-sending function that it can use to find out its current position in the document and the current editor state, and optionally produce an action to send to the editor’s onAction callback.

NodeView: interface

A node view is an object that acts as an intermediary between an editor view and the DOM representation of a certain node. An instance is created when the node is drawn, and it can handle certain aspects of the DOM behavior for the node.

  • dom: dom.Node
    The DOM node that should be used as the node’s representation.

  • contentDOM: ?dom.Node
    The DOM node that holds the node’s child nodes. Defaults to the main dom property. Only relevant for non-leaf node types.

  • parseRule: ?() → ParseRule
    Can be used by the view to control the parsing of the DOM.

  • update: ?(newNode: Node) → bool
    An optional method that the redraw algorithm will use to update the view when the node or its content changes. When present, it will be called when a DOM update finds a changed node of the same type in the place of the view’s node. When it returns true, it should have updated the DOM structure to show the changed node. It may return false to indicate that it can’t update, and a redraw should happen.

  • select: ?()
    When present, this will be called when the node is selected as a node selection. It replaces the default selected-node styling.

  • deselect: ?()
    When present, and select is also present, this will be called when the node stops being selected. It should remove any styling that select added.

EditorView.nodeViewAtPos: (number) → ?NodeView

Find the node view for the node at the given position, if any.

I’ve put my implementation of CodeMirror-for-code-blocks in a gist. It’s not super polished yet, but before I spend more time on it, I wanted to ask for feedback. If you think this wouldn’t quite work for your use case, let me know, and we can try to improve it. If you think this is a terrible idea for some reason, speak up too, since I’m not yet committed to this.

One thing that this doesn’t really solve yet is that it can be awkward to express all the state of such a nested control with node attributes. Some really transient things you can just keep as local state in the node view, but there are situations (such as when an unhandled DOM input event occurs) where ProseMirror will redraw a node just to be sure its DOM is still coherent, and you could lose that state in such a situation. But since these views are transient things rendered for document nodes, the fact that their state is determined by the node might be unavoidable.

You also get things like the need to make something, like the code block in the demo, a leaf node that would be more ‘naturally’ modeled as a textblock node. The main reason for this is that you want to manage its content yourself, and ProseMirror assumes that it gets to draw and manage actual child nodes. We could create an exception to that, but that doesn’t fix the issue that we’d now have document positions that point at things that aren’t actually managed by ProseMirror, so it couldn’t put the cursor there or find their coordinates, even though they are valid positions in the document’s range. My conclusion was that forcing a somewhat unnatural modeling is a lesser evil than having to work around such cases.

(I’m even wondering whether all nodes should be treated like this, and the editor should keep a data structure that describes the whole rendered DOM, rather than attaching properties to the DOM itself. The upside would be that a node having a view would no longer be a special case, and it would make some DOM inspection and update mechanisms faster and easier. It does, however, involve defining and maintaining yet another data structure.)

2 Likes
2

Is there a live demo where we can see this working with codemirror? Would this work if we wanted an editor that only appeared on click? For example, there was a code block but only when you selected it, it turned into a code mirror component (or a LaTeX equation that turned editable when you clicked it). I’m assuming this could just be managed internally by the NodeView when a ‘select’ action is passed.

What happens in a ‘redraw’ of a node, is the node view thrown away and then created again? In the codemirror instance this would it spin up a new codemirror object? That might cause some lag and potentially memory leaks depending on the library.

3

I’ll set up a live instance of the code next week.

Yes, that should be relatively easy to arrange, with a click handler on your node view.

Yes. You can use the onUnmountDOM prop (which might end up becoming a method on node views) if your component leaks when simply removed from the DOM.

4

Great to see you working on this!

I think that there are quite some problems that come with this approach when working in a real-time collaborative context. Instead of leveraging ProseMirror’s position mapping capabilities this approach would handle editing with “last writer wins”. Additionally we would need to send the whole code block content over the wire after every keystroke if we want to have an auto-save feature or sync the changes with other users in real-time.

What about allowing the node view to control its own DOM node, but at the same time being able to handle its content like a ProseMirror text block (instead of putting the content into the node’s attributes). For the code block an approach could look like this:

  • Subscribe to CodeMirror’s change event and translate it into a ProseMirror transform, which should update the text content of the code block node. Send this transform in the form of a NodeViewTransformAction to the onAction handler. I haven’t worked with CodeMirror yet, but I think this would involve translating line numbers and offsets from CodeMirror into absolute offsets in the ProseMirror document. We could use CodeMirror’s beforeChange event to prevent CodeMirror from immediately applying the changes.
  • The NodeViewTransformAction should be applied like any other transform and update the EditorState. Additionally it needs to be handled by the NodeView that generated this action.
  • The NodeView gets the chance to handle the action so that it can update its view to reflect the latest state. This is where we would need to translate the ProseMirror transform back into a CodeMirror transform which we could then apply to the CodeMirror instance.

The main goal of this would be to make it work in a real-time collaborative context. It’s only a rough sketch and I haven’t thought about handling selections yet as I don’t know how CodeMirror handles selections when applying changes.

Could this be a viable approach to make the example work in a real-time collaborative context? I would really love to see support for real-time collaborative editing even when integrating externally controlled views into a ProseMirror editor.

5

(I’ve uploaded a built version of the demo here. It’s still relatively flaky, consider it a proof-of-concept, not much more.)

1 Like
6

This is a good point. Collaboration would sort of work with the current approach, but it there’d definitely be rough edges, such as group undo not really working (you’d roll back other people’s changes when undoing). Actually building something that has a coherent selection and text model between the node views and the outer editor would make it possible to model managed content just like regular content, but it’d be challenging.

Setting up CodeMirror in this way wouldn’t be all that hard. The tricky part is the way ProseMirror would treat the DOM in such a managed node. But solving that problem might help lead us towards a more powerful way to represent ProseMirror’s handling of the editable DOM, so it’s definitely worth exploring.

2 Likes
7

If NodeView only has select and deselect, how does the demo handle the cursor going “up” from the top of the CodeMirror block and “down” from the end?

8

It has custom handler for the arrow keys in CodeMirror that move the selection back to the wrapping ProseMirror instance when the cursor would go out of the code block.

9

Just wanted to say overall, this would be a welcome change to ProseMirror. The collaboration to me seems ‘good enough’, not every component is going to want to maintain selections and such, worst case it could ‘lock out’ so that only one user could select the component at once.

My main concern is the redrawing of nodes, I think it can be safely assumed that redraws of NodeViews are usually going to be significantly more expensive than redraws of normal nodes.

The way we do React components in ProseMirror right now is basically generating IDs per node (though a hash of the properties may also work) and keeping the DOM elements in a hash table mapping to these IDs. When a redraw happens we look up the DOM element and return that while updating properties if needed (usually we have a wrapper DIV that’s created so that the actual react div is a child of the ProseMirror Node’s div).

Perhaps that or something like that could be implemented more broadly to reduce redraws on NodeViews?

10

That’s the main motivation behind this feature. The update method gives the node view control over redraws, so that it can do that minimally/efficiently. Full redraws can happen, but should be rare enough to not be a performance concern.

11

Some other things that I’ve been running into, the solution to which might be rolled into the work this feature:

  • When mouse clicks on a piece of DOM should be handled by ProseMirror, and when they should be left alone.

  • Strictly incremental redrawing, on the DOM node level instead of the document node level, in an attempt to make spell checking reliable (see here)

  • Moving bookkeeping info out of the (slow) DOM into a separate data structure

I’m planning to make this the priority for 0.14 (0.13 will be coming out Real Soon Now).

1 Like
12

This work has landed in release 0.14.0, roughly like I originally described it. But I did change to an approach where the editor internally uses view objects throughout to manage its DOM content, which allowed me to improve re-rendering efficiency, clean up the DOM (no more cm-... attriubutes, less wrapping spans, and mark wrappers (like <em>) can now span multiple child nodes.

You can see the docs here, and there’s now a demo using CodeMirror for code blocks live on the website.

13

@marijn great work with the node views! Seems you are on the right track with implementing custom elements. Something that I referred at Schema API design

So can we now use the node-views to define fully our own custom elements? maybe with special design-time/runtime rendering and custom inline inspectors.

I like your codemirror example - but maybe a more generic example will be to implement a simple photo gallery/slider as inline custom element. When being edited the gallery will present custom ui (in it simplest form just be able to drag new images. Something that is like for example https://github.com/orthes/medium-editor-insert-plugin for https://github.com/yabwe/medium-editor

So a gallery is a good example of self contained element. However there are some other elements that only need to be extended. For example if you are editing some Bootstrap framework elements - you only need to add custom classes to existing html elements.