Hello! I built Gem, a plain-text editor based on Prosemirror. It’s fast and minimal, and the plans are to keep it that way. I made this post because I wanted to share it, but also wanted to ask some general questions:
I want markdown-like syntax highlighting in Gem. What is the best way to set that up? Right now I have simple input rules that check whether text is *x* or _x_ and bolds or italicizes. However, I want the styles to remove themselves if the user goes back and deletes the closing tag. Its also fragile to check when content is pasted.
I.e. what I really want is to run something like a syntax highlighter that runs on the document and goes around highlighting it after transactions.
Are there other editors based on prosemirror that do this?
Is it better to create separate nodes for this (p, heading, etc.) or just attach attributes or marks to p node?
If the syntax highligher has to re-run on the whole document every time something gets typed (that’s not how I want it to work, but suppose thats the only way I can figure it out), how can I diff the new AST w/ the old AST so that the syntax doesn’t flicker when the view updates?
Right now I have simple input rules that check whether text is *x* or _x_ and bolds or italicizes.
I have seen Curvenote (see introductory post), implement what you are asking where you press $ to show the inline math editor and click on it again bring the $ chars.
Are there other editors based on prosemirror that do this?
I have been working on https://bangle.io (the editor part is open sourced at https://bangle.dev) which is rich note taking editor that runs on top of locally available markdown files.
Is it better to create separate nodes for this (p, heading, etc.) or just attach attributes or marks to p node?
I would strongly suggest for semantic reasons to use different nodes.
Updates are atomic, so it’s rather hard to accidentally introduce flickering behavior in ProseMirror. I’m not sure which AST you mean here though, so I’m not sure what kind of approach you’re planning. Something like syntax highlighting, which styles content while leaving the actual document unchanged, is best done with decorations, not nodes/marks.
Hey! Thanks for replying. Yeah the difference here between mine and your approach is that you remove the markup after styling (so remove the stars once *x* gets bolded). Once its removed, its treated as a different node, and conversion of the node requires backspace at the beginning.
Leaving the markup in place is different because the user can manipulate the markup i.e. delete the * which should result in the bold mark being removed or delete one of the # in a heading, leading to a different level heading. That’s why this is much more like syntax highlighting than it is rich-text editing.
Hmm, so
transaction is about to be dispatched => syntax highlighter runs, and creates a new AST with where highlights should be => AST gets converted to a new document state by me => I need to use view.updateState(newState) to update the state to the one the AST returned by me.
For a more concrete example:
user types *, finishing *x => syntax highlighter runs, realizes this text now needs to be bold, gives me a new intermediate AST for document => I use this AST to create a new doc state w/ {text: "*x*", marks: ["bold"]} (or something similar) => I call view.updateState
(this is a simple example that can be solved by input rules, but there can be something more complex like user pastes content that completes the formatting to text already in the document)
Does view.updateState avoid the flickering? If not, I need to either find/make a syntax highlighter that gives me transactions instead of full ASTs or diff the intermediate AST to create a transaction that I can then dispatch w/ prosemirror.
Is that more clear? Are decorations a good use case here (I am not super familiar w/ decorations)?
If you’re only doing markdown syntax highlighting, I would think it’d be overkill to use ProseMirror as opposed to CodeMirror (see CodeMirror: Markdown mode), since implementing syntax highlighting via decorations and integrating an external AST to produce those decorations based on document state is a bit hairy with ProseMirror, but comes out of the box with CodeMirror.
Use ProseMirror if you have a need for rich-text nodes and marks.
Hey Tanishq, as @bhl mentioned, CodeMirror might be perfect for your use case.
I think CodeMirror 6 also combines some great improvements to the overall API of CodeMirror (including TypeScript-first support). There’s a lot of overlap between ProseMirror’s document model and CodeMirror’s as well, so you might consider both.
The other big advantage of something like CodeMirror is customizable cursors are much less finicky.
Side note: If you check out CM6, maybe check out https://lezer.codemirror.net/ – it’s a sort of Tree-Sitter-like syntax highlighter. It’s a bit newer, but it could be fun to play with if you’re playing around.
Yep! I worked on moving to codemirror 6 today, and it took me like an hour and I was able to strip like over half the code . I do need to build a syntax highlighter, but the cursor and styling is done. Here’s the branch: GitHub - moonrise-tk/gem at codemirror
Lezer is exactly what I am looking for as well! Thanks! I was thinking of using tree-sitter and compile to wasm for syntax highlighting (I know, overkill, but I’m just playing and wanted to learn rust/wasm anyway). I might do both and see if there are performance benefits…
. I do need to build a syntax highlighter, but the cursor and styling is done. Here’s the branch: