Replicating Typora's Inline/Display Math Editing

[benrbray]

I’m attempting to use ProseMirror to replace my current note-taking workflow, which involves constantly switching between Typora/Overleaf/OneNote, with a single text editor capable of supporting blocks of text with semantic meaning.

My notes are full of inline and display math, written in LaTeX notation, so it is important that math be treated as a “first-class object” in my editor. I would like to replicate Typora’s math editing behavior as closely as possible using ProseMirror.

Typora-style Math Editing

Desired behavior:

• Property A: Show math as LaTeX code while the cursor is inside the math node. As soon as the cursor leaves the node, render using KaTeX
• Property B: Changing the cursor position by clicking / arrow keys should seamlessly transition between these two modes.

Below are some screenscaptures demonstrating the desired behavior in Typora (click to expand).

creating display math

editing display math

creating inline math

editing inline math

My Attempts

Let’s focus on inline math for now. Reading through the documentation / examples, it seems like NodeViews are the right tool to use. I have tried two approaches:

Attempt #1: Using contentDOM

Set a contentDOM in the NodeView, so that ProseMirror is responsible for managing the appearance / editing of the NodeView, as in the . This successfully achieves Property B (cursor movement with arrow keys behaves as expected), but I cannot get Property A to work (toggling katex/code views).

The reason seems to be that NodeViews with a contentDOM have no way of knowing when the editing cursor has left the NodeView. (they only receive setSelection events, not selectNode and deselectNode).

Attempt #2: Without contentDOM (demo)

Without contentDOM, the NodeView is responsible for its own rendering / editing, as in the footnotes example. Setting atom:true in the schema for my inline math node ensures that selectNode and deselectNode are called when the user selects/deslects the NodeView, allowing me to toggle the inline math editor as needed.

So, this approach succeeds at Property A (toggle math editor), but unfortunately I am struggling to implement Property B, as the cursor is always set to the first position in the NodeView, regardless of which side I approach from:

cursor approaching from the left (good)

cursor approaching from the right (bad -- cursor should enter from the right)

Attempt #3: Plugin with arrowHandler

I have also tried adapting the arrowHandler from the CodeMirror example to detect from which side the cursor enters / leaves the NodeView. However, I’m not sure how to pass this information along to the NodeView. I’ve seen other posts suggest using decorators for this purpose, but I’ve not yet been able to make that work.

Minimal Working Example

I set up a minimal working example (matching attempt #2) on codesandbox.

(if you have trouble getting the ProseMirror instance to appear, try typing a bit in the codesandbox editor to get it to refresh!)

Other Relevant Discussions

Regarding math in ProseMirror:

• Creating NodeView with Editor with InputRules
• ProseMirror + Math at Desmos

Regarding decorations to communicate with NodeViews:

• How can I communicate from a plugin to a custom NodeView?
• Updating NodeViews on node moved
• How to handle events inside decorations?
• Expanding / Collapsing view content

Miscellaneous:

• CodeMirror integration and selections
• How to create non-editable text segments in the document
• Cursor appears inside inline node when at end of preceding text node
• How can I attach attribute with dynamic value when new paragraph is inserted?

Questions

I’m still new to ProseMirror, so in addition to replicating the desired behavior, I also want to know if my solution follows ProseMirror “best practices”.

• How should I modify Attempts #1, #2, or #3 to achieve the desired behavior?
• If there are multiple solutions, what is the recommended way to achieve this behavior? Attempt #1, #2, or #3, or something else entirely?
• Based on other discussions, it seems like many people have tried to add math support to their ProseMirror editors. Unfortunately most askers seem to vanish after finding a solution to their problems! Do you know of anyone who has implemented similar behavior and posted their code publicly?

Thanks so much for this toolkit! I’m still learning, but I’ve really enjoyed working with ProseMirror so far!

[denis]

This was an issue that I ran into when developing the math+text feature for Desmos. (BTW we do not use contentDom in our implementation, but use MathQuill for WYSIWIG latex editing).

The solution for us was to listen to PM transactions using the EditorView dispatchTransaction property. On every transaction, we look at the new selection in the EditorState, and save a piece of state on each NodeView, telling it whether the current selection is before or after the NodeView.

Then, when selectNode is triggered, we know which direction we approached the node from.

    const proseMirrorView = new EditorView(node, {
      dispatchTransaction: (tr: ProseMirror.Transaction): void => {
        proseMirrorView.updateState(this.proseMirrorView.state.apply(tr));
        // update all nodes using proseMirrorView.state
      },...
    })


class MathView implements NodeView {
...
  // triggered on EditorView.dispatchTransaction
  updateCursorPos(state: ProseMirror.EditorState) {
    const pos = this.getPos();
    const size = this.node.nodeSize;
    const inPmSelection =
      state.selection.from < pos + size && pos < state.selection.to;

    if (!inPmSelection) {
      if (pos < state.selection.from) {
        this.cursorPos = 'end';
      } else {
        this.cursorPos = 'start';
      }
    }
  }

  selectNode() {
    if (this.cursorPos == 'start') {
      this.mathquill.moveToLeftEnd();
    } else {
      this.mathquill.moveToRightEnd();
    }

    this.mathquill.focus();
  }

[benrbray]

Thanks for the quick reply! I’ve created a new codesandbox based on your suggestion. I managed to get it working as expected!

Instead of calling MathQuill, I simply set the selection on the inner editor during setNode(),

let pos:number = (this.cursorPos == "start") ? 0 : this.node.nodeSize-2;

this.innerView.dispatch(
	innerState.tr.setSelection(
		TextSelection.create(innerState.doc, pos)
	)
);

However, I wasn’t quite sure how to call updateCursorPos() from the outer view’s dispatchTransaction. By default, ProseMirror doesn’t seem to expose a list of active NodeViews, so it’s not possible to e.g. iterate over them and call updateCursorPos() on each one.

The best I could come up with was below, where I maintain my own list of NodeViews that I create. However, passing around these closures feels dirty to me, so I’m wondering if you took the same approach, or if you know a better way?

This method also updates every NodeView in the document every time the selection changes, which may cause performance issues if there are hundreds or thousands of math blocks. However, we can probably avoid calling updateCursorPos() on each one by e.g. keeping them sorted by position and calling it only on the ones we know have changed after each transaction.

// editor setup
let nodeViews:InlineMathView[] = [];
let view = new EditorView(editorElt, {
	state,
	nodeViews: {
		"inlineMath" : (node, view, getPos) => {
			console.log("createMathView", nodeViews);
			let nodeView = new InlineMathView(
				node, view, getPos as (()=>number),
				// this is the hacky bit!
				()=>{ nodeViews.splice(nodeViews.indexOf(nodeView)); }
			);
			nodeViews.push(nodeView);
			return nodeView;
		}
	},
	dispatchTransaction: (tr: Transaction):void => {
		view.updateState(view.state.apply(tr));
		for (let mathView of nodeViews){
			mathView.updateCursorPos(view.state);
		}
	}
})

and the corresponding modifications to NodeView,

// custom NodeView
export class InlineMathView implements NodeView {
	// ...
	cursorPos:"start"|"end" = "start";
	onDestroy:(()=>any)|undefined;
	// ...
	constructor(node: ProsemirrorNode, view: EditorView, getPos: (() => number), onDestroy?:()=>any) {
		// ...
		// this callback should unregister the nodeview
		if(onDestroy){
			this.onDestroy = onDestroy.bind(this);
		}
		// ...
	}
	updateCursorPos(state: EditorState){
		// same as suggested method
	}
	destroy() {
		// ...
		if (this.onDestroy) this.onDestroy();
	}
}

[marijn]

@benrbray

The reason seems to be that NodeViews with a contentDOM have no way of knowing when the editing cursor has left the NodeView.

They don’t, but they do know which node decorations are active on their node. So you could have a plugin that maintains a “the cursor is in this math node” decoration, and respond to that in the node view.

[benrbray]

Thanks @marjin, I attempted to implement your suggestion in this codesandbox. The plugin / decoration /nodeview code is all in src/math-nodeview.ts, with the plugin reproduced below.

const checkCursorInMathNode = (arg: {
  selection: ProseSelection;
  doc: ProseNode;
}): IMathRenderPluginState => {
  if (!arg.selection || !arg.doc) {
    return { decorations: null };
  }
  let { $from, $to } = arg.selection;

  // only show math editor when selection entirely in math view
  if (!$from.sameParent($to)) {
    return { decorations: null };
  }
  // parent should be math_inline or math_block
  let parentName = $from.parent.type.name;
  if (!parentName.startsWith("math_")) {
    return { decorations: null };
  }

  return {
    decorations: DecorationSet.create(arg.doc, [
      Decoration.node(
        $from.before(),
        $to.after(),
        { class: "math-edit-active" },
        { isEditing: true }
      )
    ])
  };
};

export const mathRenderPlugin: ProsePlugin = new ProsePlugin({
  state: {
    init(partialState: EditorState) {
      return checkCursorInMathNode(partialState);
    },
    apply(tr: Transaction, oldState: EditorState) {
      if (!tr.selection || !tr.selectionSet) {
        return oldState;
      }
      return checkCursorInMathNode(tr);
    }
  },
  props: {
    decorations(state: EditorState) {
      return (this as ProsePlugin).getState(state).decorations;
    }
  }
});

Unfortunately, the left/right arrow keys still misbehave. The contentDOM itself (containing the LaTeX math code) should be visible only when the cursor is inside. But it seems that when computing where to place the cursor after left / right are pressed, contenteditable doesn’t know to ignore the math block and place the cursor inside the contentDOM. See below or try for yourself in the linked demo.

example of cursor issues with contentDOM

I tried to remedy this with an arrowHandler similar to the CodeMirror example, but no luck so far. (see Line 88 of src/math-nodeview.ts).

By comparison, here is a different demo of my solution using NodeViews w/out contentDOM, based on the suggestion by @denis. (and a gif of it working).

It would be great to get the contentDOM approach working correctly, since that would avoid 1) having nested ProseMirror instances and 2) manually maintaining a list of active math NodeViews. Any further help is appreciated!

[denis]

Nothing too clever here - I made an EditView transaction subscription manager. When NodeViews are created, they subscribe to listen to transactions. They unsubscribe upon being destroyed.

Our documents don’t get terribly long, so we weren’t too worried about maintaining a list of all NodeViews and updating them all, since their number usually stays below 10/doc

[marijn]

@benrbray

are pressed, contenteditable doesn’t know to ignore the math block and place the cursor inside the contentDOM

ProseMirror relies on native browser cursor motion in most cases. Setting the math block to contenteditable=false might help a bit, though I’m not sure.