Thanks @marjin, I attempted to implement your suggestion in this codesandbox. The plugin / decoration /nodeview code is all in src/math-nodeview.ts, with the plugin reproduced below.
const checkCursorInMathNode = (arg: {
selection: ProseSelection;
doc: ProseNode;
}): IMathRenderPluginState => {
if (!arg.selection || !arg.doc) {
return { decorations: null };
}
let { $from, $to } = arg.selection;
// only show math editor when selection entirely in math view
if (!$from.sameParent($to)) {
return { decorations: null };
}
// parent should be math_inline or math_block
let parentName = $from.parent.type.name;
if (!parentName.startsWith("math_")) {
return { decorations: null };
}
return {
decorations: DecorationSet.create(arg.doc, [
Decoration.node(
$from.before(),
$to.after(),
{ class: "math-edit-active" },
{ isEditing: true }
)
])
};
};
export const mathRenderPlugin: ProsePlugin = new ProsePlugin({
state: {
init(partialState: EditorState) {
return checkCursorInMathNode(partialState);
},
apply(tr: Transaction, oldState: EditorState) {
if (!tr.selection || !tr.selectionSet) {
return oldState;
}
return checkCursorInMathNode(tr);
}
},
props: {
decorations(state: EditorState) {
return (this as ProsePlugin).getState(state).decorations;
}
}
});
Unfortunately, the left/right arrow keys still misbehave. The contentDOM itself (containing the LaTeX math code) should be visible only when the cursor is inside. But it seems that when computing where to place the cursor after left / right are pressed, contenteditable doesn’t know to ignore the math block and place the cursor inside the contentDOM. See below or try for yourself in the linked demo.
example of cursor issues with contentDOM
I tried to remedy this with an arrowHandler similar to the CodeMirror example, but no luck so far. (see Line 88 of src/math-nodeview.ts).
By comparison, here is a different demo of my solution using NodeViews w/out contentDOM, based on the suggestion by @denis. (and a gif of it working).
It would be great to get the contentDOM approach working correctly, since that would avoid 1) having nested ProseMirror instances and 2) manually maintaining a list of active math NodeViews. Any further help is appreciated!