Hi all. I’ve been doing some work on delivering the promised feature of custom document schemas, along with a nice API for defining them, that I want to share with you. What I’m going to outline in this post exists in the master branch right now. It isn’t to be considered stable (further work might make sweeping changes), but it is solid enough to start a discussion on.
A document schema is a description of the shape a document can have. I am aiming to define a schema system that is powerful enough to describe most common types of documents, but not to overengineer it or complicate it so much that it becomes hard to think about. That means that, while I’m definitely interested in examples of useful documents that it can’t express, I am not committing to trying to support everything you come up with.
In the current code, each editor has a document schema associated with it. This is an object that contains a map from node names to node type objects, and another object mapping inline style name to inline style type objects.
A node type object describes something like paragraph nodes or ordered list nodes. They may describe a set of attributes, and if they do, each instance of such a node will have those attributes associated with them. An example of an attribute is the order attribute on an ordered list (describing the number at which the list starts) or the src attribute on an image element. Node types also describe the ‘categories’ of the node, which is an array of strings, and the category type it may contain. For example, the top-level document node can contain any block type. Paragraphs, ordered lists, and so on have category ‘block’. Ordered lists themselves can contain only the list_item category, which only list item nodes have. Inside of paragraphs, only inline nodes may appear, and inside of a horizontal rule or an image node, nothing may appear.
This is the way the structure of the document is constrained, and all the document transformations make sure to respect these constraints. I am working out an additional feature, where a node type can be set to be ‘fixed’, meaning normal operations can’t change its content. This has some hairy repercussions (for example, what happens when you copy part of a fixed node), but would allow modeling something like an image node that contains a caption (but not two captions, etc).
Style types may also have attributes, such as href on the link style. Other than that, they don’t do much.
Both node types and style types are instances of a given class. Defining a schema is done by mapping node names to such type classes. The classes come with default categories, contains fields, and attributes, but a specific schema can override these to reconfigure existing node types into a somewhat different structure, or to add new attributes.
There are three ‘basic’ types of nodes that all concrete node types inherit from: Block, for nesting blocks, Textblock, for things like paragraphs that are blocks, but contain flat inline content, and Inline for inline content such as images or hard breaks. (Text is a subclass of Inline with some special behavior.)
The node/style type classes are also the place where serialization and parsing logic for those nodes and styles lives. How exactly this works differs per serializer/parser. For example, the DOM/HTML serializer expects a .serializeDOM method on the node type (each node has its type object stored in a property), which it can simply call to serialize a node of that type. The DOM parser, on the other hand, gathers all the parseDOM properties of the types in the schema, which contain objects like {tag: "p", parse: ...}, telling it that whenever it encounters a <p> tag, it should call the parser that it found on that node type (in this case, the Paragraph type). Node and style types can specify handlers multiple tag names (for example <b> and <strong>), and can specify a precedence, to do something like have a high-precedence parser that checks whether a given attribute is present, and declines to handle the node if it isn’t, leaving it to a lower-precedence parser.
Other things that should end up on these type objects (but aren’t yet) are node-type-specific commands, key bindings, menu items, etc.
A schema is derived from a ‘schema spec’, which is currently a pair of two objects, the map from node names to node types and the map from style names to inline style types. For example, the definition for the default schema looks like this:
const defaultSpec = new SchemaSpec({
doc: Doc,
blockquote: BlockQuote,
ordered_list: OrderedList,
bullet_list: BulletList,
list_item: ListItem,
horizontal_rule: HorizontalRule,
paragraph: Paragraph,
heading: Heading,
code_block: CodeBlock,
text: Text,
image: Image,
hard_break: HardBreak
}, {
em: EmStyle,
strong: StrongStyle,
link: LinkStyle,
code: CodeStyle
})
All of these types are exported (from the model module), for reuse and extension. The objects actually end up looking like {text: {type: Text} – i.e. the constructors are wrapped in objects with a type property – and you can update a schema spec by calling its updateNodes (or updateStyles) method:
const flatSchema = defaultSpec.updateNodes({
list_item: {contains: "flat_block"},
blockquote: {contains: "flat_block"},
paragraph: {category: "flat_block block"},
code_block: {category: "flat_block block"}
})
This doesn’t replace these nodes, but adds adds the category flat_block to the paragraph and code block nodes, and makes the nesting block nodes (list item and blockquote) only allow that category, so that the resulting document can’t nest arbitrarily anymore.
Inside updateNodes, you can set a property to null to delete a node type, or provide a new value with type property to replace one. There’s a similar method updateStyles for messing with the styles in a schema. These are indended to make it possible to easily derive slightly changed schemas from existing schemas.
The Schema constructor takes a schema spec and produces an actual schema object (the thing you give to an editor, or use to directly create nodes). It has methods like node and text to create nodes or text nodes, and nodeFromJSON to deserialize a node in this schema that’s represented as JSON. Each node or style type instance has a link back to the schema it belongs to, so if you only have a node, you can do .type.schema to get at its schema.
I’m now thinking about integrating UI stuff into the schema system, but before I work on that, I want to address a bunch of problems with the current UI (mostly around manipulating nested content and opaque block elements). This involves rethinking a lot of stuff so it might be a while before that moves forward. Do use that time to give me feedback on this schema model.