What is the best way to replace emojis with custom inline image format?

For the current editor I am building, I would like to replace unicode emojis with images similar to other sites like Twitter and Facebook.

My current approach is to use the appendTransaction for plugins, but I am not sure if I can safely inspect the transaction details.

In the plugin, I can inspect a list of transactions by looking at each steps. AFAIK, only ReplaceStep and ReplaceAroundStep inserts text, so I then invert all steps up to the first Replace{Around}Step that inserts any unicode emojis, then re-apply all the steps again but replacing all the unicode emojis in the fragment with my custom emoji inline atomic node format.

This all seem to work pretty well. It should be fast since the code will only look at text that are changed, and it also seems pretty safe since it will catch any changes where unicode emojis may be inserted. The only problem however, is that I need to inspect step.slice which is not available in the official typescript type definition, nor is it in the official documentation. This makes it look like private API which I would like to avoid.

Is step.slice actually private and I should avoid using? Are there alternatives for reading information on a step and creating a new one? Or is this not the right approach at all?

The best way to figure out which regions a step replaced is to iterate over its step map (stepMap.forEach). You may also not need to invert and replay the steps—iterating over the new ranges (the stepmap’s ranges mapped forward over any further steps that come after them) and checking those for the pattern you’re interested in should make it possible to just directly patch only the parts of the document that need updating.

Have you considered just leaving them as plain-text emoji in the document, and having a plugin replace them with image widgets through decorations? Text seems to be a suitable format for representing emoji.

Could this be achieved with an input rule?

Input rule was one of the idea I may go with if this one doesn’t work. The problem is that input rule only works for typed out content, so I will need to also handle paste (and not sure if there are other exceptions?) Updating the transaction seems safer.

But I like the idea of using decoration. Indeed this should be a purely view level thing. We should still represent emojis as unicodes, but try to display them differently.

Good point, I suppose I should really think of input rules as another form of keyboard shortcut.

So I was able to get this to mostly work. My approach involves setting up 2 decorations around the emoji - the first one is a widget decoration before the emoji text which displays a <img> tag, and the second one is an inline decoration wrapping the emoji text in a span, which I can then use css to make disappear so that the image emoji becomes what’s displayed.

However, the inline decoration is a bit tricky. There’s no way to determine the actual width of the emoji, and in css, I can’t set a width if the span is display: inline. So I had to set it as display: inline-block which then allows me to set the width and a negative margin + opacity 0 to make it ‘disappear’. The inline block screws up the cursor management somewhat, so I have to also set contentediable=false on the span, which works most of the time. The only exception is when the span is the last text node in a line - in that case there’s no place for the cursor to be placed so the cursor disappears on Firefox, and moves to the end in Safari.

I made an issue here for this behavior: https://github.com/ProseMirror/prosemirror/issues/1069 Any advices here?

If you are curious about other approaches I have tried:

  1. display: inline, as mentioned, doesn’t allow setting width so I can’t reliably make the span disappear with nagative margin
  2. font-size: 1px makes the cursor also 1px on Chrome
  3. display: none also screws up the cursor since there’s no place to place the cursor (the span node doesn’t really ‘exist’ when it has display: none)