toggleMark variant to prevent toggling the mark in atom's content (e.g. footnotes)

1

My editor provides footnotes, in a way similar to the footnotes example in Prosemirror documentation.

When you select a text around the footnote marker and toggle a Mark, say bold, the Mark gets toggled not only on the selected span of the main text, but all over the footnote content as well. Luckily you can check that even in the footnotes example.

I wrote a toggleMark variant that can leave the footnote’s content untouched.

That behavior is controlled through two new options in the third argument of toggleMark:

  • excludeAtomsContent: exclude only the content of atoms, while the atoms themselves get the Mark toggled (in the example above, the note marker would be bold, while the note text would not)

  • excludeAtomsWithContent: exclude whole atoms that have a content (atoms that accept no content – isLeaf === true – are not excluded)

Here’s the code:

export function toggleMark(markType: MarkType, attrs: Attrs | null = null, options?: {
  /// Controls whether, when part of the selected range has the mark
  /// already and part does not, the mark is removed (`true`, the
  /// default) or added (`false`).
  removeWhenPresent: boolean,
  /// Ignore the content of selected atoms
  /// (Atoms get the mark toggled, their content does not).
  excludeAtomsContent?: boolean,
  /// Ignore selected atoms that have content.
  excludeAtomsWithContent?: boolean
}): Command {
  let removeWhenPresent = (options && options.removeWhenPresent) !== false
  return function (state, dispatch) {
    if (!markType) return false
    let { empty, $cursor, ranges } = state.selection as TextSelection
    if ((empty && !$cursor) || !markApplies(state.doc, ranges, markType)) return false
    if (dispatch) {
      if ($cursor) {
        if (markType.isInSet(state.storedMarks || $cursor.marks()))
          dispatch(state.tr.removeStoredMark(markType))
        else
          dispatch(state.tr.addStoredMark(markType.create(attrs)))
      } else {
        let add, tr = state.tr
        if (removeWhenPresent) {
          add = !ranges.some(r => state.doc.rangeHasMark(r.$from.pos, r.$to.pos, markType))
        } else {
          add = !ranges.every(r => {
            let missing = false
            tr.doc.nodesBetween(r.$from.pos, r.$to.pos, (node, pos, parent) => {
              if (missing) return false
              missing = !markType.isInSet(node.marks) && !!parent && parent.type.allowsMarkType(markType) &&
                !(node.isText && /^\s*$/.test(node.textBetween(Math.max(0, r.$from.pos - pos),
                  Math.min(node.nodeSize, r.$to.pos - pos))))
            })
            return !missing
          })
        }

        // code to exclude atom's content
        let sranges = ranges
        const excludeAtomsContent = options?.excludeAtomsContent
        const excludeAtomsWithContent = options?.excludeAtomsWithContent
        if (excludeAtomsContent || excludeAtomsWithContent) {
          const { doc, selection } = state
          const { from, to } = selection
          const excluded: number[][] = []
          doc.nodesBetween(from, to, (node, pos) => {
            if (node.isAtom && !node.isLeaf) {
              let cFrom: number, cTo: number
              if (excludeAtomsContent) {
                cFrom = Math.max(pos + 1, from)
                cTo = Math.min(pos + 1 + node.content.size, to)
              } else {
                cFrom = Math.max(pos, from)
                cTo = Math.min(pos + node.nodeSize, to)
              }
              excluded.push([cFrom, cTo])
            }
          })
          sranges = excluded.reduce(
            (acc, [eFrom, eTo]) => {
              const newAcc: number[][] = []
              acc.forEach(([aFrom, aTo]) => {
                if (eFrom >= aTo || aFrom >= eTo)
                  newAcc.push([aFrom, aTo])
                else if (eFrom > aFrom && eTo < aTo)
                  newAcc.push([aFrom, eFrom], [eTo, aTo])
                else if (eFrom < aFrom)
                  newAcc.push([eTo, aTo])
                else
                  newAcc.push([aFrom, eFrom])
              })
              return newAcc
            },
            ranges.map(({ $from, $to }) => [$from.pos, $to.pos])
          ).map(([from, to]) => new SelectionRange(doc.resolve(from), doc.resolve(to)))
        }
        // end of code to exclude atom's content

        for (let i = 0; i < sranges.length; i++) {
          let { $from, $to } = sranges[i]
          if (!add) {
            tr.removeMark($from.pos, $to.pos, markType)
          } else {
            let from = $from.pos, to = $to.pos, start = $from.nodeAfter, end = $to.nodeBefore
            let spaceStart = start && start.isText ? /^\s*/.exec(start.text!)![0].length : 0
            let spaceEnd = end && end.isText ? /\s*$/.exec(end.text!)![0].length : 0
            if (from + spaceStart < to) { from += spaceStart; to -= spaceEnd }
            tr.addMark(from, to, markType.create(attrs))
          }
        }
        dispatch(tr.scrollIntoView())
      }
    }
    return true
  }
}

I have some doubts about the options argument:

  • should removeWhenPresent become optional (boolean | undefined)?

  • maybe a single option excludeAtomsWithContent: "whole" | "content" could be better than two boolean options

2

I realized the implementation is wrong, since it prevents applying marks in footnotes at all.

I’ll be back with a working implementation.

3

This is better:

export function toggleMark(markType: MarkType, attrs: Attrs | null = null, options?: {
  /// Controls whether, when part of the selected range has the mark
  /// already and part doesn't, the mark is removed (`true`, the
  /// default) or added (`false`).
  removeWhenPresent: boolean,
  /// Ignore the content of selected atoms
  /// (Atoms get the mark toggled, their content does not).
  excludeAtomsContent?: boolean,
  /// Ignore selected atoms that have content.
  excludeAtomsWithContent?: boolean
}): Command {
  let removeWhenPresent = (options && options.removeWhenPresent) !== false
  return function (state, dispatch) {
    if (!markType) return false
    let { empty, $cursor, ranges } = state.selection as TextSelection
    if ((empty && !$cursor) || !markApplies(state.doc, ranges, markType)) return false
    if (dispatch) {
      if ($cursor) {
        if (markType.isInSet(state.storedMarks || $cursor.marks()))
          dispatch(state.tr.removeStoredMark(markType))
        else
          dispatch(state.tr.addStoredMark(markType.create(attrs)))
      } else {
        let add, tr = state.tr
        if (removeWhenPresent) {
          add = !ranges.some(r => state.doc.rangeHasMark(r.$from.pos, r.$to.pos, markType))
        } else {
          add = !ranges.every(r => {
            let missing = false
            tr.doc.nodesBetween(r.$from.pos, r.$to.pos, (node, pos, parent) => {
              if (missing) return false
              missing = !markType.isInSet(node.marks) && !!parent && parent.type.allowsMarkType(markType) &&
                !(node.isText && /^\s*$/.test(node.textBetween(Math.max(0, r.$from.pos - pos),
                  Math.min(node.nodeSize, r.$to.pos - pos))))
            })
            return !missing
          })
        }

        // code to exclude atom's content
        let sranges = ranges
        const excludeAtomsContent = options?.excludeAtomsContent
        const excludeAtomsWithContent = options?.excludeAtomsWithContent
        if (excludeAtomsContent || excludeAtomsWithContent) {
          const { doc, selection } = state
          const { from, to, $from } = selection
          const depth = $from.sharedDepth(to)
          const shared = $from.node(depth)
          if (shared) {
            const start = $from.start(depth)
            const relFrom = from - start
            const relTo = to - start
            const excluded: number[][] = []
            shared.descendants((node, pos) => {
              if (node.isAtom && !node.isLeaf) {
                let cFrom: number, cTo: number
                if (excludeAtomsContent) {
                  cFrom = start + Math.max(pos + 1, relFrom)
                  cTo = start + Math.min(pos + 1 + node.content.size, relTo)
                } else {
                  cFrom = start + Math.max(pos, relFrom)
                  cTo = start + Math.min(pos + node.nodeSize, relTo)
                }
                excluded.push([cFrom, cTo])
              }
            })
            sranges = excluded.reduce(
              (acc, [eFrom, eTo]) => {
                const newAcc: number[][] = []
                acc.forEach(([aFrom, aTo]) => {
                  if (eFrom >= aTo || aFrom >= eTo)
                    newAcc.push([aFrom, aTo])
                  else if (eFrom > aFrom && eTo < aTo)
                    newAcc.push([aFrom, eFrom], [eTo, aTo])
                  else if (eFrom < aFrom)
                    newAcc.push([eTo, aTo])
                  else
                    newAcc.push([aFrom, eFrom])
                })
                return newAcc
              },
              ranges.map(({ $from, $to }) => [$from.pos, $to.pos])
            ).map(([from, to]) => new SelectionRange(doc.resolve(from), doc.resolve(to)))
          }
        }
        // end of code to exclude atom's content

        for (let i = 0; i < sranges.length; i++) {
          let { $from, $to } = sranges[i]
          if (!add) {
            tr.removeMark($from.pos, $to.pos, markType)
          } else {
            let from = $from.pos, to = $to.pos, start = $from.nodeAfter, end = $to.nodeBefore
            let spaceStart = start && start.isText ? /^\s*/.exec(start.text!)![0].length : 0
            let spaceEnd = end && end.isText ? /\s*$/.exec(end.text!)![0].length : 0
            if (from + spaceStart < to) { from += spaceStart; to -= spaceEnd }
            tr.addMark(from, to, markType.create(attrs))
          }
        }
        dispatch(tr.scrollIntoView())
      }
    }
    return true
  }
}

Now, when the selection is all inside a footnote, you can toggle the mark.

When the selection covers part of the main text and the whole footnote or part of it, only the main text portion gets the Mark toggled, while the footnote text is untouched.

4

I think the excludeAtomsWithContent option is too specific. If anything, that should be a generic. This patch implements enterInlineAtoms in a somewhat more succinct way.

1 Like
5

Thank you, @marijn !

I do agree, enterInlineAtoms is better.

I saw you also fixed markApplies to consider that option (I saw the problem too, but I did not clearly understand how to fix it).

In my code I have another option that controls whether the Mark is applied to the footnote node or only to its content. In the footnote example that would likely influence only the style of the footnote marker.

In my editor, though, I have marks for inline quotes, and a plugin that adds couples of delimiters (e.g. “”, «», ‘’) around them, using appendTransaction().

So think of an inline quote – I’m highlighting it with italic – that contains a footnote (“1” is the footnote marker):

Paragraph with a quote that contains1 a footnote.

The effect of the automatic delimiters plugin is

Paragraph with «a quote that contains1 a footnote».

when the quote Mark is set on the footnote node;

but it becomes:

Paragraph with «a quote that contains»1« a footnote».

when it’s not set on the footnote node.

That’s why, in my code, I had to add an option to control setting the Mark on the atom node or only to its content.

6

This is how my implementation behaves by default (and what I expect will be the appropriate behavior in most situations).