Discussion: The limits of actions and reducers

[marijn]

In 0.11, ProseMirror moved to a linear dataflow architecture, where the editor state isn’t imperatively updated, but only changes in response to actions that must be explicitly applied to a state to get a new state, which must then be explicitly handed to the editor view.

This makes the library quite a bit less accessible to people not used to that style, but has had amazing effects on how easy it is to write robust extensions, and on how clean it allows the implementation of the view to be. So I’m going ahead and declaring the departure from imperative state management a resounding success.

However, I jumped into this with very little experience with these kind of systems, and assumed that if I just did what Elm and Redux did, everything would fit into place. It’s not quite turning out that way—the theory behind this kind of architecture isn’t all that mature yet, and ProseMirror has a number of requirements that make it harder to apply this stuff than it is in TodoMVC (or, in fact, most web apps).

The main complicating requirement is modularity. Most of the literature around Elm or Redux assumes that the action types and reducers that act on a given piece of state are designed and written as a single coherent module. But ProseMirror’s plugin architecture means that plugins can define new action types, add state fields with their own reducers, and may need to respond to actions that influence other parts of the state (the document or selection, usually).

These are some issues I or users of the library have been running into:

• Since the reducer for state.doc is provided by the library, and because other reducers have to be able to recognize transforms, only the transform action type can update the document. This means that undo/redo actions or collaborative editing updates have to mask themselves as transform actions.

• That also means that an action can’t be both a document transform and update the set of stored marks (the marks that are applied the next time you type). In general, actions can’t be composed in this approach, which really gets in the way when writing plugins.

• Another thing we’re not really providing in a safe way at the moment is a way to follow up on actions with some kind of invariant-preserving check. You an wire your onAction callback to inspect the action and apply another action if it wants to, but you can’t do this at a lower level (except sort of, with the horrifically hacky extendTransformAction), you can’t do it from a plugin, and if you do multiple such things, it’s hard to ensure that they don’t interact in problematic ways.

I’ve been thinking hard about how to adjust the way actions work in ProseMirror to solve these, but I haven’t really found any acceptable approach yet. So I’m writing this to lay out the problems, sketch some possible directions, and ask for feedback or links to relevant resources.

Requirements

These are the existing features that I think should be preserved:

• An editor view is completely defined by its current state, and updating it is just a matter of giving it a new state.

• It must remain possible to extend the editor state with new fields from plugins.

• Reducers, as well as code that routes actions, must be able to reason about actions. That means it must be able to recognize effects that the action has, at least on parts of the state that the code knows about.

• Related to the previous point—it must remain easy to guarantee that the state remains consistent. State updates must happen in a way that makes it easy for reducers to ensure that the piece of state that they are responsible for stays in sync with other pieces of state.

• Preserving some degree of conceptual simplicity would be really nice.

These are new requirements that I’d like to support:

• Composability. It must be easy to write new action types that have the effect of existing action types among their result when applied.

• We need some way for action routing code to add more actions or more effects without doing dangerous things like extendTransformAction—extending a previously created action, which represents a coherent state update, is wrong. But what to do instead isn’t clear to me yet.

• Similarly, plugins should somehow be able to respond to actions by fixing up the state. For example to enforce document constraints that the schema can’t express, or to prevent certain actions for being applied entirely.

The following are some ideas I’ve been pursuing, along with the reasons why they might be problematic.

Batched actions

If the thing you dispatch is not a single action, but an array or other sequence structure of actions, which then all get applied to create a new state, that solves some of these problems. There’s some precedent for this in Redux-land.

This makes it easy for action dispatchers to append more actions to a batch if they wanted to.

It could also be seen as a way to dispatch multiple effects at the same time—simply precede your custom action by standard actions that have the desired effects.

But that is somewhat scary, since it breaks the atomicity of actions. What if something ends up cancelling some but not all of your actions? Boom, inconsistent state.

Also, when adding more actions to such a batch, it is not trivial to figure out the actual end state of those actions (which you’ll want to base your new actions on), since it hasn’t been computed yet. That could be worked around with…

Transaction objects

We could compute the new state right away, and dispatch the new state instead of an action. But that makes it impossible for the code that receives that state to figure out what happened.

As an alternative, we could define some kind of object that holds a new state and an array of actions. Maybe call it Transaction. You’d build it up one action at a time, and then dispatch it. The dispatch code could inspect the actions and, if it wanted to, add more, and finally call updateState on the editor view with the resulting state.

Plugins could be allowed to respond to an action being added to a transaction by cancelling the action or appending another action. It remains really tricky to do this in a way that guarantees consistency, though. See the section on action pipelines later on.

It would add one more object type to worry about, and take us further away from the other architectures in this space.

Properties as effects

Right now, only actions with a type property of "transform" can change the document. Another approach to composability would be to not make the state field reducers dispatch on type, but rather on the presence of some property. For example, a transform property would mean that this action transforms the document, a selection property would mean that it updates the selection, and so on. (We already kind of do this in the optional selection property of transform type actions.)

The idea would be to use plain old string properties for the state effects defined by the core library, and expect other modules to use symbols (or some polyfill). Thus, an undo action, instead of pretending to be a transform action with some extra properties, could have its own type, and include properties that describe the change to the document as well as the way the history should be updated for that action.

A downside is that it, too, takes us further from classical reducer architecture, and it might make the role of the type action attribute somewhat difficult to grasp for people coming from there. Rather than a tagged-union style data structure, actions would become something more like a tuple of effects, and the type would be a descriptive string hinting at the action’s origin rather than a description of its payload.

Plugins and the action pipeline

Our current plugin and state field system already makes the story of how a new state gets computed from an old state and an action somewhat more involved than it is in its purest form. Allowing plugins to directly influence the way actions are applied allows you to do some very useful things, but would complicate this even further.

As I touched on before, it’d be nice if plugins could cancel actions and respond to actions with more actions and maybe even replace actions. The question is how to do this in a way that’s not terribly error-prone.

As long as, as in the current system, a state update is a single invocation of the reducers for a single action, it is relatively clear what’s going on. Once applying an action can spawn more actions, things get subtle.

Say plugin X adds action A2 in response to action A1. But then plugin Y cancels A1 when it gets a turn, should probably cancel A2 as well, since the start state of A2 is no longer available.

What if only A2 was cancelled, but plugin X relied on that to guarantee some kind of consistency?

Maybe only the starting action should be cancellable, and once that goes through, plugins can add their followup actions. But note that if plugin X has already added A2, then if plugin Y wants to add another action in response to A1, it has to base that action on the state after A2.

Things get much more awkward once you start dealing with chunks of actions.

So then you might be tempted to think that replacing actions is a better approach than adding more actions. But updating/replacing things that you don’t fully understand (this action might be fired by some plugin that your code has never heard of and contain who-knows-what kind of properties and consistency constraints) is probably never going to be safe either.

Also, unless we implement something like the transaction data type introduced earlier, if plugins mess with actions at apply-time, the code that delivered the action (say, the onAction callback or something that called it) doesn’t get a complete view of what happened. But it might be responsible for keeping some piece of state external to the editor state consistent with the editor state, which is rather hard to do if you don’t know how the editor state was updated.

Conclusion

In conclusion, there are real issues with the current design, and addressing them appears to be hard. Yet, I’d really like to hammer this out before 1.0. So I’m going to continue obsessing over it, and if you have any ideas or resources that might be helpful, let’s hear them.

[marijn]

One I forgot to mention:

Decoupling state transitions from reducers

Instead of exposing document transforms through a reducer that directly responds to certain actions, it might help to have an applyTransform method or function which takes a state and produces a new state with a transform applied. Reducers for specific actions can then call this, and state fields may declare which types of updates they are interested in, and update themselves when such a change occurs. Different reducers could then defer to the same method, without duplicating any code or responsibility. Similar methods could be defined for selection, stored mark, etc updates.

This has downsides too.

For one, state updates are no longer single-step, because when there are multiple effects (as in a transform that updates the selection and also stores the fact that the selection should be scrolled into view on the next view update) you’d have to compute multiple new states, with some of the intermediate ones being nonsensical (selection doesn’t fit the doc).

It also makes it impossible to inspect actions and predict their effect, since that’s now encoded in an action-type-specific reducer that we don’t know about. (Maybe we should embrace this, and require code that needs to know about effects to simply diff states – though that would mean it’d only be possible to do this after the action has been applied.)

[rainhead]

I am not steeped in functional patterns, so I’m sorry I can’t be more helpful here. However, issues of shared state exist outside the Elm architecture, they’re just harder to see. While Elm and Redux are new, people have been writing complex, functionally-written programs successfully for some time. Maybe someone can chime in with a book recommendation.

You’re stuck between imperative and functional paradigms practically as well as conceptually. Rich text editing in a web browser must be one of the most frustrating places you could try to impose purity. And conceptually, you seem to be divvying up responsibilities in a way that fits better in an imperative world. A few specific things that stand out as wrong to me are:

• that actions should represent anything other than explicit actions taken by a user,
• the notion of a component canceling the behavior of any component it is not directly responsible for, and therefore,
• the need for plugins to declare shared state fields.

I’m sure it’s all in your head, but have you laid out in prose or diagrams the specific ways that plugins–you might call them aspects–need to collaborate?

[Gozala]

Hi Marijn,

I have looked into Redux, although I have not actually used it. That being said I do know Elm and in fact have being using it’s architecture in JS Introduction · reflex there are some fundamental differences between Elm and Redux so not all of my feedback maybe applicable to Redux.

Before I dive into let me just give a very brief overview of how render loop works in Elm. Each logical component is usually defined as a module (written in flow syntax):

Counter.js

class Model {
  value:number
  constructor(value:number) {
    this.value = value
  }
}

export type Message =
  | { type: "Increment" }
  | { type: "Decrement" }

export type Options =
  | number
  | void

export const init = (options:Options=0):[Model, Cmd<Message>] => {
  return [new Model(options), Cmd.none]
}

export const update = (message:Message, model:Model):[Model, Cmd<Message>] => {
   switch (message.type) {
      case "Increment":
        return increment(model)
      case "Decrement":
        return decrement(model)
    }
}

export const view = (model:Model):DOM<message> => {
  // ...
}

export const main = program({ init, update, view })

This may seem unrelated but I’ll draw a connection after an example

Now in Elm if you want to create alternate version of counter you usually wrap existing one and provide alternate view or maybe functionality.

For instance if you want to override create counter that also has a reset you’d do following:

AltCounter.js

import * as Counter from "./Counter"

class Model {
  counter:Counter.Model
  constructor(counter:Counter.Model) {
    this.counter = counter
  }
}

export type Message =
  | { type: "Reset" }
  | { type: "CounterMessage", counterMessage: Counter.Message }

const counterMessage = (message:Counter.Message):Message =>
  { type: "CounterMessage", counterMessage: message }

export const init = (options:Counter.Options):[Model, Cmd<Message>] => {
  const [counter, counterCmd] = Counter.init(options)
  return [new Model(counter), counterCmd.map(counterMessage)]
}

export const update = (message:Message, model:Model) {
   switch (message.type) {
      case "Reset":
        return init()
      case "CounterMessage":
        const [counter, counterCmd] = Counter.update(message.counterMessage, model.counter)
        return [new Model(counter), counterCmd.map(counterMessage)]
    }
}

export const view = (model:Model):Html<Message> =>
  Html.div(null, [
     Counter.view(model.counter).map(counterMessage),
     Html.button([onClick(reset)], [Html.text("Reset")])
  ])

Essentially AltCounter has a full control of the embedded Counter. Which means it can infer every action from update and decide to apply it, ignore it, or substitute it instead. It can in fact apply the update look at the state and decide to tweak it further or revert back to previous version. I am not going to explain details of Cmd but that could be used to schedule extra effects that in result will send a message:Message back into update, which gives you yet another opportunity to cancel message but send a different one instead.

We pretty much exploit this architecture to provide plugins system in browser.html. Unfortunately code to follow is fairly long (mainly due to polymorphic nature of it) but the gist of it is that you can wrap any app component (which I’ll refer to as Host in the code) with a PluginManager (see code below) to allow each registered plugin:

1. Infer a Host messages before they are applied and update own state in effect.
2. Infer a Host message and before they are applied and enqueue additional message(s) to a Host in effect. That could be used by plugin to tell a host to do something extra or maybe undo (there for cancel) stuff that will be done by Host once message is applied.
3. Infer a Host message and apply it to it’s model to check what changes would it cause and as a result do 1 or/and 2.
4. Allow rendering of it’s own view.

PluginManager.js

type PluginMessage <ownMessage, hostMessage> =
  | { type: "OwnMessage", ownMessage: ownMessage }
  | { type: "HostMessage", hostMessage: hostMessage }


type Plugin <hostMessage, hostModel, ownMessage, ownModel, ownConfig> = {
  id: string,
  config: ownConfig,
  unload(state:ownModel, host:hostModel):ownConfig,
  load(options:ownConfig, host:hostModel):[ownModel, PluginMessage<ownMessage, hostMessage>],
  update(msg:PluginMessage<ownMessage, hostMessage>, state:ownModel, host:hostModel):[ownModel, Cmd<PluginMessage<ownMessage, hostMessage>>],
  view(model):Html<PluginMessage<ownMessage, hostMessage>>
}

type InstalledPlugin <hostMessage, hostModel, ownMessage, ownModel, ownConfig> = {
  plugin: InstalledPlugin<hostMessage, hostModel, ownMessage, ownModel, ownConfig>,
  options: ownConfig,
  instance: null | {state: ownModel}
}

type Plugins <hostMessage, hostModel> =
  {[key:string]: InstalledPlugin<hostMessage, hostModel, *, *, *>}

class Model <hostMessage, hostModel>  {
  plugins: Plugins<hostMessage, hostModel> 
  host: hostModel
  constructor(host:hostModel, plugins:Plugins<hostMessage, hostModel>={}) {
    this.host = host
    this.plugins = plugins
  }
}

type Message <hostMessage, hostModel, pluginMessage, pluginModel, pluginConfig> =
  | { type: "InstallPlugin",  installPlugin: Plugin<hostMessage, hostModel, pluginMessage, pluginModel, pluginConfig> }
  | { type: "UnloadPlugin", unloadPlugin:  string }
  | { type: "LoadPlugin", loadPlugin: string }
  | { type: "MessagePlugin", pluginID: string, pluginMessage:PluginMessage<pluginMessage, hostMessage> }
  | { type: "MessageHost", message: hostMessage }

const hostMessage = <hostMessage, hostModel> (message:hostMessage):Message<hostMessage, hostModel, *, *, *> =>
  ({ type: "MessageHost", message }

const pluginMessageFrom = (id:string) => <pluginMessage, hostMessage, hostModel> (message:pluginMessage):Message<hostMessage, hostModel, pluginMessage, *, *> => {
  switch (message.type) {
    case "OwnMessage":
      return { type: "PluginMessage", pluginID: id, pluginMessage: message.ownMessage }
    case "HostMessage":
      return { type: "HostMessage", hostMessage: message.hostMessage }
  }
}

export const init = <hostOptions, hostModel, hostMessage> (host:Host<hostOptions, hostModel, hostMessage>, options:hostOptions):[Model<hostMessage, hostModel>, Message<hostMessage, hostModel>] => {
  const [host, cmd] = host.init(options)
  return [new Model(host, {}), cmd.map(hostMessage)]
}

export const update =<hostMessage, hostModel> (updateHost:(message:hostMessage, model:hostModel)=>[hostModel, Cmd<hostMessage>, message:Message<hostMessage, hostModel>, model:Model<hostMessage, hostModel>):[Model<hostMessage, hostModel>, Cmd<Message<hostMessage, hostModel>>] => {
   switch (message.type) {
      case "PluginMessage":
        const {pluginID, pluginMessage} = message
        const installedPlugin = model.plugins[pluginID]
        if (installedPlugin == null) {
          return [model, Log.error(`Plugin ${pluginID} is not installed`)]
        } else if (installedPlugin.instance == null) {
         return [model, Log.error(`Plugin ${pluginID} is not enabled`)]
        } else {
          const {plugin, options, instance} = installedPlugin
          const [state, cmd] = plugin.update(message.pluginMessage, instance, model.app)
          const plugins = {...model.plugins, [pluginID]: {plugin, options, instance: {state}} }
          return [
            new Model(model.host, plugins)
            cmd.map(pluginMessageFrom(pluginID))
          ]
        }
      case "HostMessage":
        const cmds = []
        let state = model
        const pluginMessage = { type: "HostMessage", hostMessage: message.hostMessage }
        for (let pluginID in model.plugins) {
          const [model, cmd] = update({ type: "PluginMessage", pluginID, pluginMessage: message }, model)
          state = model
          cmds.push(cmd)
        }
        const [host, cmd]  = updateHost(message.hostMessage, state.host)
        cmds.push(cmd.map(hostMessage))
        return [new Model(host, state.plugins), Cmd.batch(cmds)
    }
    // ....
}

export const view = <hostMessage, hostModel, pluginMessage, pluginModel, pluginConfig> (viewHost:(model:hostModel)=>Html<hostMessage>, model:Model<hostMessage, hostModel>):Html<Message<hostMessage, hostModel, pluginMessage, pluginModel, pluginConfig>> =>
Html.div(null, [
  viewHost(model.host).map(hostMessage),
  ...[...Object.entries(model.plugins)].map(([id, plugin]) => plugin.instance == null ? "" : plugin.view(plugin.instance.state).map(pluginMessageFrom(id))
])

This is how proposed solution meets declared requirements:

• An editor view is completely defined by its current state, and updating it is just a matter of giving it a new state.

Nothing really changes here.

• It must remain possible to extend the editor state with new fields from plugins.

Proposed solution is slightly different. Instead of adding new fields to the state plugin added state lives separately. That being said proposed solution could be modified to use host[Symbol.for(pluginID)] for storing plugin states instead, but it has drawback of not being very type checker friendly and plugins could temper with each other, but maybe for your use case that is a better option.

• Reducers, as well as code that routes actions, must be able to reason about actions. That means it must be able to recognize effects that the action has, at least on parts of the state that the code knows about.

This seems somewhat specific to Redux as far as I can tell. I assume proposed solution could be adapted to Redux and if so, here is how above solution addresses that:

1. Rather than making plugins hook at the root level and make them pattern match over all possible messages / actions variants, kinds of plugins could target specific host components and there for be only concerned with messages / actions and state of that component.

2. Plugin could in fact import the Host module and perform Host.update(msg.hostMessage, host) to inspect the effects that app will have once that message is applied (internally Host.update could be optimized to cache say last return value so that actually work is only performed once per update cycle).

• Related to the previous point—it must remain easy to guarantee that the state remains consistent. State updates must happen in a way that makes it easy for reducers to ensure that the piece of state that they are responsible for stays in sync with other pieces of state.

I am sure I fully understand this requirement. But here is what you get by proposed solution that I think is relevant:

1. Plugins maintain their own state and update it as necessary on own or host messages. They can’t mess with host state other than sending messages that host can handle, there for they can’t put host in an invalid state as long as host never returns one in response to a message.
2. Plugins are able to update own state on every host (or own) message, which means they can stay in sync. Also since other plugins can only send host a messages all plugins will synchronize once it is received.
3. Example does not illustrate this but you could do rollback if one of the plugins throws exception or maybe unload that plugin & attempt to process message again.

• Preserving some degree of conceptual simplicity would be really nice.

Apart from the highly polymorphic PluginManager I think architecture is pretty simple. Plugins just:

• Receive own and / or host messages.
• Send self / host messages.
• Update own state when messages are received.
• Provide a view to render own state (you could exploit that better at the individual component level than illustrated)

These are new requirements that I’d like to support:

• Composability. It must be easy to write new action types that have the effect of existing action types among their result when applied.

You could do it to some degree. You can define your OwnThing message / action and once received update your own state and also send HostThink message to perform existing action. Provided example would perform send messages in the followup update (but in the same render cycle) but you could change that if you like, I would not recommend though as in that case you pretty much sacrifice with state consistency as that means any plugin could update host state before other plugins get to handle the message / action.

• We need some way for action routing code to add more actions or more effects without doing dangerous things like extendTransformAction—extending a previously created action, which represents a coherent state update, is wrong. But what to do instead isn’t clear to me yet.

This is too Redux specific. If proposal does not address this maybe you could describe this requirement in non Redux specific terms so I could provide feedback.

Similarly, plugins should somehow be able to respond to actions by fixing up the state.

That could be done by sending a message / action to a host to make it fixing up a state.

For example to enforce document constraints that the schema can’t express, or to prevent certain actions for being applied entirely.

Proposal could be adjusted to allow preventing action from being applied entirely, although that would come at the cost of “conceptual simplicity”. For example you could update Plugin.update return type to:

[ownModel, Cmd<PluginMessage<ownMessage, hostMessage>>, boolean]

Where 3rd item in the tuple is true if you’d like to prevent action, in which case PluginManager update could just return [model, cmd.map(pluginCommandFrom(pluginID)] back essentially rolling back any own state updates that plugins performed prior to cancelling along with commands they queued, instead performing a command returned by plugin that cancelled it.

I hope this helps!

[frederik]

@Gozala the plugin architecture makes a lot of sense to me. Especially the part where Plugins that create error states can be ignored / unloaded.

Adding actions that manipulate state in batch seems dangerous to me since it will be hard to reason about the state (an effect would not have that problem since it only modifies state after the original action has gone through - but then we’re stuck with “incomplete” states of course).

Allowing the plugins to manipulate state and then letting the host emit the final state can counteract that nicely (if I understand this correctly). It kind of seems a bit like zone.js will wait for all my asynchronous actions to complete and handle errors on the way.

@marijn Is there a specific issue we could use for argument’s sake? At most points I can grasp the issue but it’s hard to discuss on such a level without a lot of experience with all the different concepts involved.

[kofifus]

I wonder if the vuex (‘redux’ for Vue.js) architecture can be helpful. . state can be divided into modules and instead of actions they have mutator to change state… see their plugin framework at https://vuex.vuejs.org/en/plugins.html

[marijn]

@rainhead

that actions should represent anything other than explicit actions taken by a user,

This is pretty much what our actions represent—changes to the editor state caused by user actions. They do describe them in the vocabulary of the state (document/selection) not the input (key/mouse) because

1. we often need to allow the browser’s default behavior for user actions, so ‘dispatching’ has to happen right away, and feed back into whether to call preventDefault
2. changes may be read from the way the DOM structure and selection changed, rather than directly from events, in which case they are naturally closer to the editor state level than the event level (though we do ‘raise’ some types of DOM changes back to something more like an intent, doing this for all possible ones would be too complex)
3. modeling actions as specific user intents would lead to a ton of different action types, which again makes it hard to conclude anything based on the action value

@rainhead

the notion of a component canceling the behavior of any component it is not directly responsible for

The main use case here is things like disallowing certain types of edits. It is possible that this would be better modeled at a different level (before the change gets enshrined in an action), but I haven’t found an attractive approach yet.

@rainhead

the need for plugins to declare shared state fields.

Plugins need to have state, and need their reducers to be invoked at the same time as those of the main editor state, so putting that state into the same object works well. A more purist approach could require them to wrap the state, but then you’d need to unwrap every time you want to access the actual, basic state fields like document and selection, and that’d just be really awful to program against.

[marijn]

@Gozala Thanks for the write-up.

@Gozala

This seems somewhat specific to Redux as far as I can tell.

Note that ProseMirror is in no way related to Redux (it doesn’t depend on anything like that, and the architectural influence of Elm was stronger than that of Redux). If the word ‘reducer’ is where you go that idea from, you can mentally replace it with ‘update function’ if you want.

So anywhere where you say something is too Redux-specific, I’m not sure what that means.

One aspect of your approach appears to be that only host messages are allowed to change the host state. But with plugins like the undo history or collaborative editing, plugins must be able to send plugin-specific messages that both have a custom effect on the plugin’s state, and update the host state.

You mention sending a host message, but I’m A) not sure how that would work (an update function can’t send a message) and B) very worried about the lack of atomicity that’d cause (which you also point out).

[marijn]

@frederik

@marijn Is there a specific issue we could use for argument’s sake? At most points I can grasp the issue but it’s hard to discuss on such a level without a lot of experience with all the different concepts involved.

These are the specific cases that are currently difficult:

• Defining commands that wrap other commands, for example a command that extends another command by auto-joining any adjacent lists of the same type it produced (to be used with, say, deleteSelection and lift). This could be done by wrapping the onAction callback passed to the inner commands and doing something to the actions traveling through there. This one’s nice to have, but I’d be willing to drop it and find another approach if an otherwise good solution doesn’t allow it.

• Keeping a document-dependent data structure on the side, outside the editor state. For example, the change tracking demo keeps a blame map describing the responsible commits for each range of the document. This has to be updated every time the document is transformed, which is currently done by looking at the actions that are being applied, and updating the map for transform actions.

• Having a plugin fire custom actions that modify both some built-in piece of state (say, the document) and a plugin-specific piece of state. I.e. if the collaborative editing plugin gets an update from the server, it needs to change the document and also its current understanding of when it last synced. When this happens, other plugins should be able to notice the change to the document.

• Allowing a plugin to somehow mess with document transforms. If I have a table node that allows colspan/rowspan cells, the restriction that tables have to be rectangular can no longer be applied at the schema level. But I’d still like to immediately patch up tables that have the wrong number of cells in a row. How this should be modeled, I don’t know yet.

[ThariqS]

I don’t have particular insights into the patterns, but I have been running into similar problems while writing plugins.

I have been getting into the mindset of ‘everything is a plugin’ and it has certainly simplified development, but like you noted there are some frustrating limits. Particularly, the ability to update the editor state within a plugin, right now decorations are essentially the only way that a plugin can really effect the document.

That said, do you anticipate these changes to mostly extend the existing plugin model or have a lot of breaking changes? Would like to know before investing time into the current model. Will it strongly effect any of the API outside of the plugins?

[marijn]

@ThariqS

That said, do you anticipate these changes to mostly extend the existing plugin model or have a lot of breaking changes?

I haven’t settled on anything yet, so I can’t say. I’d prefer something low-impact, obviously.

[marijn]

Here’s something that might work:

Actions to _Trans_actions

Drop the concept of an ‘action’ as a raw object with a type property that suggests some kind of user action. Define a Transaction class that fills its role instead, while also replacing the EditorTransform class. (Note that this isn’t the Transaction type I proposed in the first message.)

These are objects that represent any kind of atomic change in the editor state. They are still applied, much like actions currently are, but instead of being interpreted by reducers they rather explicitly declare the kind of changes they embody. (This is actually already the case for the current actions, but their name suggests otherwise, and making terminology correspond to function is helpful.)

Transactions are class instances that have a fields corresponding to the basic properties of the editor state (doc, selection, storedMarks), but the doc property holds a Transform object (probably to be renamed to DocTransaction) instead of a direct document. They also know their starting state (and you get an error when trying to apply them to another state), automatically track the timestamp at which they are created and allow some descriptive fields like an origin string.

Plugins each get a ‘slot’ in a transaction into which they can put plugin-specific data. When a transaction is applied, each plugin gets a chance to update its plugin state based on the transaction object, just like they do now.

Before applying a transaction, plugins get a chance to reject it. If no plugin rejects it, it is applied, and afterwards plugins get a chance to follow up with another transaction. If one does, it is applied, and all plugins again get a chance to follow up. The result of applying a transaction is a new state and an array of transactions that actually ended up being applied (so that the applying code can find out what happened).

Why does this help

Most importantly, it moves from the awkward, not quite appropriate (for this system) concept of actions to something that more clearly evokes what these things do: transactions over the editor state. The advantages of passing these around as first-class values are still there, without suggesting they are something they aren’t (user actions).

• An editor view is completely defined by its current state

• It must remain possible to extend the editor state with new fields from plugins

• Reducers, as well as code that routes actions, must be able to reason about actions much easier and cleaner now

• It must remain easy to guarantee that the state remains consistent

• Preserving some degree of conceptual simplicity I think this is easier to explain

• Composability. Effects on the state are now separate fields in a transaction, and not tied to action types anymore

• Some way for action routing code to add more actions or more effects. Not entirely covered, but this might be a bad idea. It would be possible for transactions to know whether they can be safely extended, but I haven’t decided whether I want to go there.

• Plugins should somehow be able to respond to actions by fixing up the state. Covered in the transaction-application protocol.

[ThariqS]

It sounds useful but a bit hard to grasp without seeing what the API changes would look like.

Is one Step equivalent to one Transaction? Or can multiple Steps be bundled into a Transaction, like actions currently are.

[marijn]

@ThariqS

Is one Step equivalent to one Transaction? Or can multiple Steps be bundled into a Transaction, like actions currently are.

Multiple steps. They work like Transform objects.

Basically, actions are gone and the EditorTransform is extended to fill that same role. I’ve been doing some coding and the required changes are very mechanical so far (remove .action calls when dispatching a change, renaming onAction to dispatch, and changing what used to be custom actions to add custom properties to a transaction instead.)

[marijn]

I’ve pushed some changes that implement the new design, and updated the website demos. Here’s what the release notes would look like if I did a release now:

prosemirror-state

Breaking changes

The way state is updated was changed. Instead of applying an action (a raw object with a type property), it is now done by applying a Transaction.

The EditorTransform class was renamed Transaction, and extended to allow changing the set of stored marks and attaching custom metadata.

New features

Plugins now accept a filterTransaction option that can be used to filter out transactions as they come in.

Plugins also got an appendTransaction option making it possible to follow up transactions with another transaction.

prosemirror-view

Breaking changes

The onChange prop has been replaced by a dispatchTransaction prop (which takes a transaction instead of an action).

The handleDOMEvent prop has been dropped in favor of the handleDOMEvents (plural) prop.

New features

Add view method dispatch, which provides a convenient way to dispatch transactions.

The dispatchTransaction (used to be onAction) prop is now optional, and will default to simply applying the transaction to the current view state.

Added support for a handleDOMEvents prop, which allows you to provide handler functions per DOM event, and works even for events that the editor doesn’t normally add a handler for.

[marijn]

So after porting the demos I think this is a keeper. The code got slightly simpler (no need to call .action() anymore, more straightforward reducers), and some of the more demanding cases, like the message flow in the collab editing demo, got easier to express — for example, I can now pack information about remote annotation changes, meant for the annotation managing plugin, into the transaction that represents remote editing changes, which are received in the same request.

I’ll update the docs and put out a new release, probably tomorrow. Still very interested in feedback.

[ThariqS]

This looks great, minimal changes. I’m assuming the serialization format of steps and diffs (with the collab plugin) will remain the same?

While we’re talking about Plugins, I wanted to give some feedback on usecases I’ve run into. One thing I’ve noticed is that there is significant book keeping to do for fairly simple tasks.

For example, if you want to put in a decoration on a certain set of nodes, these decorations must be updated on every transform through mappings or finds, this feels a lot like boilerplate, but maybe all that’s needed is some good helpers.

Also, communication between nodeviews and plugins is kind of difficult. A plugin can really pass information to a node view via decorations (and thus requires the same book keeping) or the nodeviews can access plugins through the getPlugin interface.

[Gozala]

@marijn

Note that ProseMirror is in no way related to Redux (it doesn’t depend on anything like that, and the architectural influence of Elm was stronger than that of Redux). If the word ‘reducer’ is where you go that idea from, you can mentally replace it with ‘update function’ if you want.

My bad, I presumed so due to word reducer and actual mention of Redux.

If the thing you dispatch is not a single action, but an array or other sequence structure of actions, which then all get applied to create a new state, that solves some of these problems. There’s some precedent5 for this in Redux-land.

So anywhere where you say something is too Redux-specific, I’m not sure what that means.

I guess only case it is really applicable was here:

We need some way for action routing code to add more actions or more effects without doing dangerous things like extendTransformAction—extending a previously created action, which represents a coherent state update, is wrong. But what to do instead isn’t clear to me yet.

I assumed that extendTransformAction was Redux specific, but maybe it’s a ProseMirror specific. It is just that requirement is really tide to some specific routing mechanism & not knowing it makes it hard to tell if proposed solution addresses it or not.

One aspect of your approach appears to be that only host messages are allowed to change the host state. But with plugins like the undo history or collaborative editing, plugins must be able to send plugin-specific messages that both have a custom effect on the plugin’s state, and update the host state.

Yes idea is that plugins manage their own state & if they need to also update host state they accomplish that by sending a host a message (that presumably host understands).

You mention sending a host message, but I’m A) not sure how that would work (an update function can’t send a message)

I tried my best to show that with an actual code, although I guess it may have assumed certain level of familiarity with Elm where update returns [Model, Cmd<Msg>] pair where Cmd<Msg> is an effect that will send Msg back into the update loop. PluginManager.js shows intimate details of how this could be utilized to allow plugins to send messages to the “host”.

and B) very worried about the lack of atomicity that’d cause (which you also point out).

I am not sure what do you mean. Sorry.

For what its worth, I think approach you settled on is in fact not that different from Elm’s or what I was trying to outline. From what I understand Transaction is closer to what “message” types are in Elm than actions were although it’s one generic curated type vs union of many. Main difference from what I understand is that instead of sending a followup messages from plugins to host, each plugin gets a chance to alter or cancel transaction before it is applied. Which is a neat way to batch things up without running into possibly recursive update cycle.

API aside only conceptual difference is that in my proposed approach plugin did not get to see effects submitted by plugins that run prior to it before host message was handled. Instead each plugin could submit a followup effect via message to a host that would get handled through another update cycle.

In chosen approach plugin gets to see effects submitted by plugins that run prior to it & can contribute or reject along with a host effect (or rather transform).

I think chosen approach is really neat! It does introduce certain unpredictability due to order of plugin execution though. Unlike in solution proposed plugin A may never have a chance to reject change submitted by plugin B as later will run later and subsequently get applied to the document. It is hard for me to say if this is good or bad in practice though as plugins counteracting each other seems like a problematic setup anyhow.

P.S. I’d love to see a post about the chosen approach as both Elm & Redux communities could learn from it.

[marijn]

@Gozala

From what I understand Transaction is closer to what “message” types are in Elm than actions were although it’s one generic curated type vs union of many. Main difference from what I understand is that instead of sending a followup messages from plugins to host, each plugin gets a chance to alter or cancel transaction before it is applied.

It’s rather that transactions can contain plugin-specific extra messages, which are handled (by the plugin) at conceptually the same moment as the rest of the transaction, so that the whole update (core state + plugin state) happens in one shot. This was somewhat inspired by the plugin message/core message split in your example.

Plugins can also follow up on transactions with a separate transaction, but that’s intended for handling responses to transactions not created by the plugin itself (as opposed to, say, an undo transaction which the history plugin itself created).

@Gozala

very worried about the lack of atomicity that’d cause (which you also point out).

I am not sure what do you mean. Sorry.

Whenever an update is spread out over multiple transactions, there will be intermediate states in which some part of it have been applied an other haven’t. Any code looking at that intermediate state might go wrong because the separate fields of the state might not be coherent.

@Gozala

Unlike in solution proposed plugin A may never have a chance to reject change submitted by plugin B as later will run later and subsequently get applied to the document.

You have a typo there but I assume you mean that there’s a risk of a transaction slipping through due to the order in which plugins get to look at it. I specifically designed this to avoid this: A transaction only goes through if no plugin cancels it. Transactions can’t be updated, only canceled, before they are applied. After a transaction is applied, every plugin gets a chance to follow up with another transaction. The callback responsible for this gets passed an array of transactions, along with the states before and after those were applied. When a plugin decides to add a transaction, all plugins again get a chance to cancel it, and if none do, it is applied and the maybe-append-transaction process continues in a way that makes sure that every plugin is given a chance to look at every transaction that was applied. This could land you in an infinite loop, if two plugins keep responding to the other’s transactions by appending another, but I guess that’s relatively unlikely, and the guarantee that you can see every transaction is really useful.

Anyway, thanks again for taking the time to write up your solution, it really helped me define the problem better. A blog post is probably a good idea, at some point, though I’m still not sure I can properly explain why this works.

[marijn]

@ThariqS

This looks great, minimal changes. I’m assuming the serialization format of steps and diffs (with the collab plugin) will remain the same?

Yes, the changes are only in the approach to the client-side state-update loop.

If you want to discuss the plugin architecture, please open a different thread.