Create Yjs/Prosemirror updates programmatically

We’re using Yjs+Prosemirror. The Yjs updates are persisted in a database. For a data migration, we need to add marks to certain words. The migration needs to run on the backend (i.e. we can’t migrate only once a user opens the document in the frontend).

The following snippet adds the mark into the Prosemirror document:

// Construct editor state from persisted Yjs updates
const yDoc = new Y.Doc();
const prosemirrorSchema = new Schema(...);
Y.applyUpdate(yDoc, updatesFromDatabase);
const prosemirrorDoc = yDocToProsemirrorJSON(yDoc);
const node = prosemirrorSchema.nodeFromJSON(prosemirrorDoc);
const editorState = EditorState.create({
  schema: prosemirrorSchema,
  doc: node,
});

// Create transaction to add the mark
const mark = schema.markFromJSON({
  type: 'my-mark',
  attrs: { ... },
});
let tr = editorState.tr;
tr = tr.addMark(from, to, mark);

// Apply transcription to the Prosemirror state
const newEditorState = editorState.apply(tr);

Q: What I’m struggling with: How to create a Yjs update that reflects the Prosemirror transaction (which I could then persist in the database)?

The workaround we implemented so far is that we replace all existing yjs updates in the database with a new update that contains the completely migrated document.

The disadvantage of this approach that it breaks the normal history, which means:

  • It requires special logic in the frontend to handle such replacement updates (yjs can’t simply apply the update but the document state must be reset).
  • It requires a different approach in the backend to rollup updates.

You could add an update event listeners before you modify the document. If there were any change, there should be an update. If you perform more involved changes, a cleanup event could generate another transaction, so it might make sense to merge updates.

const updates =[ ]
ydoc.on('update', update => updates.push(update))

// Apply transcription to the Prosemirror state
// This change will generate a Yjs update
const newEditorState = editorState.apply(tr);

if (updates.length > 0) {
  // at least one update was generated
  //  merge all updates to one..
  const change = Y.mergeUpdates(updates)
  broadcast(change)
}

// @todo unregister ydoc update event handler

Thanks a lot, Kevin. This is a very interesting approach! Will give it a shot.

@dmonad Thanks for your tip again! I gave it a shot but it looks I’m still doing something wrong:

const yDoc = new Y.Doc();

// Captures Yjs updates triggered by Prosemirror changes
const capturedUpdates = [];
const capture = update => capturedUpdates.push(update);
yDoc.on('update', capture);

// Construct editor state from persisted Yjs updates
const prosemirrorSchema = new Schema(...);
Y.applyUpdate(yDoc, updatesFromDatabase);
const prosemirrorDoc = yDocToProsemirrorJSON(yDoc);
const node = prosemirrorSchema.nodeFromJSON(prosemirrorDoc);
const editorState = EditorState.create({
  schema: prosemirrorSchema,
  doc: node,
  plugins: [ySyncPlugin(yDoc.getXmlFragment('prosemirror'))],
});

// Create Prosemirror transaction to add mark
const mark = schema.markFromJSON({
  type: 'my-mark',
  attrs: { ... },
});
let tr = editorState.tr;
tr = tr.addMark(from, to, mark);

// Apply transcription to the Prosemirror state
editorState.apply(tr);

yDoc.off('update', capture);
console.log(`capturedUpdates=${capturedUpdates.length}`);

const yChange = Y.mergeUpdates(capturedUpdates);

This captures the update from the initialization (applying updatesFromDatabase) but it doesn’t capture the update from the add-mark transaction.

Could this be related how I set up the ySyncPlugin?

There are a few 0-second timeouts before the ProseMirror state is populated with the Yjs document. So you probably need to wait a bit before manipulating the document.

There’s no need to populate the state like this: const node = prosemirrorSchema.nodeFromJSON(prosemirrorDoc); This is going to be automatically at startup.

There’s no need to populate the state like this: const node = prosemirrorSchema.nodeFromJSON(prosemirrorDoc); This is going to be automatically at startup.

Thanks a lot @dmonad. This brought me in an interesting direction. Based on your remark, I had expected the editor state to be the same as the Ydoc state eventually:

function sleep(ms: number) {
	return new Promise(resolve => setTimeout(resolve, ms));
}

const yDoc = new Y.Doc();
const schema = new Schema(...);
Y.applyUpdate(yDoc, updatesFromDatabase);
const editorState = EditorState.create({
  schema,
  plugins: [ySyncPlugin(yDoc.getXmlFragment('prosemirror'))],
});

await sleep(100);

console.log(JSON.stringify(yDocToProsemirrorJSON(yDoc)));
console.log(JSON.stringify(editorState.doc.toJSON()));

Yet, that’s actually not the case:

{"type":"doc","content":[{"type":"paragraph","attrs":{"style":""},"content":[{"type":"text","text":"Hello world"}]}]}
{"type":"doc","attrs":{"version":"3"},"content":[{"type":"paragraph","attrs":{"nodeIndent":null,"nodeTextAlignment":null,"nodeLineHeight":null,"style":""}}]}

While Yjs contains the correct content (“Hello world”), the editor doesn’t - even after waiting for 100ms (same if I wait for 1+ sec).

Could this be caused by the code running on the backend (NodeJS) instead of in a browser?

That is because the editor instance has a schema, the other function does not. y-prosemirror doesn’t store null information, because that is just the default value. The schema enforces that all potential attributes are set.

That is because the editor instance has a schema, the other function does not. y-prosemirror doesn’t store null information, because that is just the default value. The schema enforces that all potential attributes are set.

I see how this explains the extra attributes but I’d have expected the editor state to contain also the text node {"type":"text","text":"Hello world"}]}.

Hmm… I don’t know what’s happening there. If you think this is a bug, you can open a bug report in the y-prosemirror repository.

Thanks, @dmonad ! I created a minimal reproduction case and ticket for it: Prosemirror document doesn't get populated with Yjs doc content (sync plugin) · Issue #106 · yjs/y-prosemirror · GitHub