Contenteditable is bad, but contenteditable combined with Android is even worse. Just for context, Slate (another rich text editing library) had to make a GoFundMe (Add Android Support for the Slate WYSIWYG Editor by Sunny Hirai — Kickstarter ) so a few engineers could dedicate their time to getting something plausible. Because of how bad it is, I thought I would try to document the various issues, post some of our workarounds, and list what tools we have at our disposal.
We rebuilt our entire editor from the ground up using ProseMirror. Our old app struggled with Android but with ProseMirror’s design, we were able to make giant improvements in our mobile editor. That being said, with better features comes more risk for terrible browser and OS bugs. Below are a few issues we’ve had to deal with. Some of the issues are bugs specific to ProseMirror but I would put no fault in ProseMirror itself. ProseMirror has some very effective workarounds internally but this is a game of whac-a-mole.
ProseMirror has been so good in fact that using plugins, we’ve been able to effectively build our own hacks workarounds to help solve many of these issues. We have an android input plugin that is about 450 lines long and a separate input plugin that tracks much of the input state (composing, deleting content, finishing a composition, focus, etc.).
How Android Input Works and Available Tools
Compositions and IMEs
Android primarily uses composition based input for almost everything (dom events - What is JavaScript's CompositionEvent? Please give examples - Stack Overflow). It fires similar events to typing in Japanese/Korean/Chinese on desktop and it is why you see the little underline when you are typing a word.
That little underline means that the text is currently being composed and the whole word may change as you type. These systems are called IMEs (input method editors) and the Android IME is especially annoying when combined with contenteditable. The one exception for Android is that when you use a bluetooth keyboard, the events are essentially the same as typing on desktop.
Our understanding based on digging into Android apps and staring at tons of logs is that the Android IME views the contenteditable block as one long plain text string. This is troubling when you have lots of HTML that you are editing as the IME may view your content very differently than how the user perceives it.
ProseMirror has a lot of functionality to handle compositions/IMEs and has to walk a fine line between control and not interfering with native behavior. For one, it has to protect composition text nodes while at the same time being willing to sync new changes from state updates. Even standard React controlled inputs struggle with this balance (see Change event fires extra times before IME composition ends · Issue #3926 · facebook/react · GitHub) and plain text inputs are a couple of orders of magnitude less complicated than a rich text editing input.
ProseMirror uses composition event handlers internally to help with composition issues (CompositionEvent - Web APIs | MDN) and the various event types correspond to a composition lifecycle (compositionstart, compositionupdate, compositionend).
Another important set of tool are input events (InputEvent - Web APIs | MDN). Their life cycle is typically
- A beforeinput event fires with a specific inputType
- The browser mutates the dom
- An input event fires with a specific inputType
These events are fired constantly for all sorts of various inputs and provide hints as to what the input is trying to do using the “inputType” field (and there are a lot of inputTypes Input Events Level 2). We use input events all of the time to help workaround bugs as they provide crucial hints as to what the browser is trying to do.
If you are interested in learning more about all of the events that fire when typing, I suggest checking out this site Keyboard Event Viewer on various devices and seeing how varied the events are.
Multiple Keyboards and Ways of Typing
One important note is that Android has many types of keyboards with the most popular being GBoard. However, Samsung and LG both have their own keyboards that are used by default and then there are additional third party keyboards that are commonly used (like Swift keyboard). On top of that, there are also different ways of “typing” as you can use swipe typing, voice typing, etc… These keyboards have their own unique quirks and bugs and unless you get native help via a mobile app, you can’t tell which type of keyboard is active.
A few examples
- LG keyboard often struggles with undo/redo mutations during a composition
- Samsung tends to actually include Backspace keys for keyboard events
- Gboard fires additional selection changes (maybe due to the composition text node swap in Issue number 3
And many more
Cursor Parking
This is a tool we use internally to help try to force the browser and IME to reset their internal state. I posted about it here https://discuss.ProseMirror.net/t/cursor-parking-a-tool-to-help-handle-native-browser-issues/2107 but the short version is that we move the selection (and sometimes focus) out of the ProseMirror into a temporary input, let the browser trash that input as they see fit, then move the selection back. It’s analogous to unplugging a wifi router and plugging it back in when you have issues (except without the downtime).
Issues
Issue 1: Lack of keydown keys
This is one of the biggest and most annoying issues that we’ve ever had to deal with. The crux of the issue is that Android normally does not tell you which key is pressed on keypress events (keydown, keyup, etc.). This is what a typical keyboard event on Android looks like (notice the key and keyCode fields)
This means that most of your custom bindings will not work most of the time. This is especially annoying when you have custom backspace handling (like selecting a node first before deleting on backspace).
ProseMirror has some workarounds internally to help with this by detect the changes in the mutation and looking to see if the mutation looks like a backspace or enter, but it doesn’t work 100% of the time.
We can mostly detect if a backspace/delete is happening using a beforeinput event and looking for an inputType of either deleteContentBackward or deleteContentForward. We then fire a synthetic keydown that is either a Backspace or a Delete key (respectively) and use someProp(‘handleKeyDown’). If the event is “handled” (if a handleKeyDown prop function returns true), then things get interesting.
According to the spec on beforeinput events, beforeinput is supposed to be cancelable for most inputTypes (see Input Events Level 2). Unfortunately, browsers often don’t care enough about the spec so we have to prevent the default behavior using a combination of cursor parking and suppressing selection updates. Here is a terrible function we use that hacks some internal ProseMirror goodies to help prevent this default handling.
const suppressSelectionUpdatesAndTextInsertions = (view: EditorView): void => {
(view as any).suppressTextInsertions = true;
// ProseMirror checks this flag internally and then reverts selections if a selection change fires while this is active
(view as any).domObserver.suppressingSelectionUpdates = true;
setTimeout(() => {
(view as any).suppressTextInsertions = false;
(view as any).domObserver.suppressingSelectionUpdates = false;
}, 50);
};
Issue 2: The cursor wrapper
The cursor wrapper has a few uses but one of the primary use cases is to take stored marks and make sure composition text is styled correctly. ProseMirror internally calls this use case “mark cursor.” The clearest example is you go to an empty line, you enable bold, and you start typing. The composition text should be bold.
This works because ProseMirror inserts a cursor wrapper decoration into the dom in the form an image element with no src and then selects that image element when the composition starts. The browser then overwrites that image element and keeps the styles. However, this mark cursor can cause a few problems with the Android IME.
One is that if it’s removed at just the right time, then the selection may end up jumping a single character. Another issue is when you split inclusive marks. The mark cursor decoration is added to the middle and when issue 3 occurs (see that in the next section), Android loses track of where to re-insert the composition text. Update: This is actually a general Android contenteditable issue, it isn’t specific to ProseMirror and the cursor wrapper
One last issue is that if you are tracking input states yourself in plugin state, you can unintentionally remove the cursor wrapper at the wrong time. We have input state that tracks if a composition is happening, if a composition is finishing, if content is being deleted, and a few other details. Dispatching these meta transactions in their associated event handlers can cause the cursorWrapper to be unset at the exact wrong time which will then wreak havoc in the editor. We got around this by overwriting the view getter for the cursor wrapper and checking for meta only transaction. If the transaction was only a meta transaction, then we would lock the cursor wrapper for the next state update.
const setupCursorWrapperProxy = (view: EditorView) => {
// We need to lock the cursor wrapper sometimes in case we trigger meta only updates. Otherwise, we get tons of input issues
let privateCursorWrapper: any = null;
Object.defineProperty(view, 'cursorWrapper', {
get() {
return privateCursorWrapper;
},
set(v) {
// Don't allow the cursor wrapper to be unset while locked. Setting new cursor wrappers is fair game, but not unsetting them
if (this.cursorWrapperLocked && !v) return;
privateCursorWrapper = v;
}
});
};
const isMetaOnlyTransaction = (tr: Transaction) => {
return Boolean(!tr.isGeneric && !tr.docChanged && !tr.selectionSet && !tr.storedMarksSet);
};
...
dispatchTransaction: tr => {
view.cursorWrapperLocked = isMetaOnlyTransaction(tr);
view.updateState(view.state.apply(tr));
},
Issue 3: The composition text node swap
When performing custom mutations (like a custom enter handling when splitting a paragraph), Android has this really strange behavior where it seems to want to swap out text notes with a composition text node. If you put your cursor next to some text and hit enter, Android seems do the following after the enter handling is complete (mostly synchronously)
- Select the word the cursor ended up next to
- Delete the word (beforeinput and input fire with an inputType of deleteContentBackward)
- Fire a phantom keydown for seemingly no reason
- Start a composition
- Re-insert the the word deleted at step 2 via an input event with an input type of insertCompositionText
I think some ProseMirror internals may have problems with this text node swap because selectionchange events are asynchronous so at step 1, ProseMirror’s selection internally is out of date with what the native selection. My theory is that that issue caused the problem where Android seems to “lie” about where the selection is, which can cause this issue inside of ProseMirror.
Right now, we don’t have a great workaround for this and I think the cursor jumping would need to be fixed inside of ProseMirror by syncing the selection during the deleteContentBackward beforeinput event. Then, we may want to hold off on flushing the mutations until Android finished everything.
Issue 4: Android Chrome is afraid of unselectable atoms
Let’s say you have a node that is an atom and there is nothing selectable inside of that node. What Android often does is that when you backspace onto that node, Android will just give up and dismiss the keyboard. This is quite annoying when you are trying to delete content and you come across one of these unselectable atoms. Also for some nodes (especially inline nodes), the cursor will just jump right over the node and keep deleting text (which seems is most likely due to how the IME views the document). Finally, the Android IME will often get confused when you have a NodeSelection and will then start a composition with nearby text. If you type while in this NodeSelection, sometimes weird mutations will occur and text nearby will get randomly updated.
To help “trick” Android and the IME that there is selectable content inside of a Node, we have occasionally added this selection helper element in React (as most of our NodeViews are using a React binding)
export default function SelectionHelper(props: JSX.IntrinsicElements['img'] & { inline?: boolean }) {
const { inline = false, ...rest } = props;
return React.createElement(inline ? 'span' : 'img', {
...rest,
style: {
position: inline ? 'static' : 'absolute',
display: inline ? 'inline' : 'block',
margin: '0 auto',
height: 0,
width: 0,
userSelect: 'all',
msUserSelect: 'text',
WebkitUserSelect: 'all',
MozUserSelect: 'all',
overflow: 'hidden',
...props?.style
}
});
}
We have also used some CSS class trickery to force user-select: all on unselectable items when your cursor is getting close to unselectable atoms. We also occasionally try to manually delete some content on deleteContentBackward input events (see issue #1) for inline nodes using backspace handlers.
For the NodeSelection issue, we have gotten around this using cursor parking.
Issue 5: Input bugs are unique across different webview versions, keyboards, keyboard versions, types of devices, and OS versions.
We are lucky to have a large enough QA team with a medley of different device combinations but even with that, there are still some nasty and stupid bugs that come up that are specific to a combination of certain webview versions on specific devices with specific keyboards. We are actually using our native app to tell our editor which keyboard types (e.g. Gboard, Samsung, LG, etc.) are enabled at any given time so we can have workarounds for specific keyboards if absolutely necessary. Many of our fixes in these scenarios end up using cursor parking but this is still an area where we are all bound to have blind spots.
Where to go next?
Honestly unless there are serious efforts on the Chrome front to improve contenteditable on Chrome Android and how the IME views contenteditable, there isn’t much we can do fundamentally that ProseMirror isn’t already doing. Even if we fix some of the bugs listed in this post and added some more nuance to ProseMirror’s mutation handling, I think there will continue to be issues. I’m sure there are a few additional fixes that could be made but issues like the mark cursor seem to be unfixable without breaking the mark cursor feature at least some of the time.
The best approach may be to just have less functionality on mobile, especially around key handling and advanced node views, but I’d love to hear what others think and if anyone has unique ideas to handle the myriad of issues here.
Resources
- Keyboard Event Viewer
- https://medium.engineering/why-contenteditable-is-terrible-122d8a40e480
- Feature/android keyboard support by wleroux · Pull Request #4200 · ianstormtaylor/slate · GitHub
- https://discuss.ProseMirror.net/t/cursor-parking-a-tool-to-help-handle-native-browser-issues/2107
- Input Events Level 2
- InputEvent - Web APIs | MDN
- CompositionEvent - Web APIs | MDN
