What's the correct way to set default content for y-prosemirror?

[Mirone]

Hi there, I’m struggling with set the default content of a prosemirror editor with Yjs.

First Approach

Use prosemirror’s doc property when create state.

const state = EditorState.create({
    schema,
    doc,
    plugins,
})

This is not working, Yjs won’t respect the doc property and it will erase content in the editor when sync.

Second Approach

Use prosemirrorToYDoc to create the Y.Doc, and initialize ySyncPlugin with this Y.Doc.

It can create an editor with the correct default content. However, everytime I refresh the page or let a new user join the room. The content will be copied and users will get duplicated contents.

After a new user joining the room:

Third Approach

I tried to update the editor content through editorView.dispatch(newTr) programmatically. However it will influence undo manager’s history and when user call redo command, the content will be erased. Which is not what I expect.

---

So, what’s the correct way to create an editor with default content?
I expect the user behavior as:

1. When a user join an empty room, the editor is created with default content.
2. When second user join the room, the editor content is just synced from other users.

[dmonad]

Hi @Mirone,

Only one peer should populate the document with content. Populating content is an insertion. Therefore, duplicate insertions of “default content” will always lead to duplication of content.

Your first approach doesn’t work because y-prosemirror prefers the state of the Yjs type and overrides the existing content. Otherwise, we’d also get duplication of content.

My recommendation is to only initialize a document once. This can happen on the first client that creates a document. You will avoid a lot of complicated issues if you simply keep the Yjs document around instead of re-initializing it every time you open a document.

[Mirone]

Some links might be useful:

Initial offline value of a shared document

In another discussion, I described a “templating engine” that might be relevant for your use-case. Merging changes from one document into another The idea would be to always start with a “template” that, for example, contains a headline and an empty paragraph. You could create a template and store it as a base64 in your source-code. E.g. const ydoc = Y.Doc() ydoc.getXmlFragment().insert(0, new Y.XmlElement('p')) const template = bufferToBase64(Y.encodeStateAsUpdate(ydoc)) Every time you execu…

Merging changes from one document into another

Hi all, I’ve got a scenario where I’m trying to create a document that contains content from multiple template documents. The user should be able to customize the new document as they see fit. I’d also like to be able to modify the template documents and have a mechanism for those changes to flow into the new document (based on an id key or something). I’m not quite sure how to accomplish this or if this is possible. Below is a scenario that I’m trying to get working. Any thoughts / questions …

[Mirone]

Thanks for your clarification. I’m wondering that how can I know if there’s other client that already populated contents?

[vojto]

You could ask server for all the updates, and then just check if the document is empty. If it’s empty, it means no one has initialized the content yet, and you can initialize it.

I’m dealing with the same problem, but I can’t just wait to go online, because our app needs to work offline.

So if you’re app goes online, you can just wait until connection is established.

[dmonad]

As I said, it makes more sense to design your application in a way so that only the client that creates a document populates it with content.

I often see developers working around this, trying to populate the document from a JSON representation of their data instead of simply storing the Yjs document (possibly alongside the JSON representation). This leads to all kinds of problems and complexities that you really want to avoid.

Waiting for a “sync” event is not good enough. You could have a client with an older version of the document (re)-joining the session after a short disconnection. In the best case, your re-populated content gets duplicated, in the worst-case newer changes get overwritten by the old version.

If you go this route, you need to implement some kind of session management. y-websocket does not support this. You also want to “populate” the content on the backend instead of the client. You need to have some process to elect a peer that initializes content. This can happen, for example, through a proper lock implementation (e.g. redlock - super complex, hard to really understand). Electing a client-peer, with a potentially bad network connection, will lead to problems that are impossible to debug, so choose a peer with a good network connection (server).

[Mirone]

I think here what I want to implement is something like a template. I tried to implement it like this:

// when user connect
wsProvider.once('synced', async (isSynced: boolean) => {
    if (isSynced) {
        collabService.applyTemplate(doc, template).connect();
    }
});


class CollabService {
    applyTemplate(doc, template) {
        const yDocNode = yDocToProsemirror(schema, doc);

        if (yDocNode.textContent.length === 0) {
            const templateDoc = prosemirrorToYDoc(template);
            const template = encodeStateAsUpdate(templateDoc);
            applyUpdate(doc, template);
            templateDoc.destroy();
        }
    }
}

How do you think about this solution?

[dmonad]

@Mirone This approach is still vulnerable to the duplication of content. As I explained in the original thread, you should store the base64 somewhere.

[Mirone]

If I’m making an app or website. I’ll store the base64, however, I’m working on the milkdown editor framework which is similar to tiptap but has markdown support.

So, I can recommend users to store the base64. But I also need to provide a way to make users can use markdown as template.

Also, I’m wondering what’s the difference if users store the prosemirror JSON instead of base64 of ydoc? IMO, they can be transformed into each other.

[dmonad]

If this is executed twice, you still duplicate content:

const templateDoc = prosemirrorToYDoc(template);
const template = encodeStateAsUpdate(templateDoc);
applyUpdate(doc, template);
templateDoc.destroy();

You need another way to prevent duplication of content.

Waiting for the sync event is also not good enough, because sometimes clients are just disconnected for a short period of time. Once they rejoin… BAM… duplication of content.

I hear you that you want to store everything as markdown. I’m just saying that it is highly recommended to store the Yjs document instead somewhere. Once the editing history is lost, Yjs will duplicate content.

It doesn’t matter which framework for conflict resolution you use (CRDTs, OT, Git, …), you must always retain the history, otherwise, you can’t resolve conflicts.

Content should be initialized only once by the client that initially creates a document. After that, you should keep the Yjs document laying around somewhere (indexeddb, file system, database, wherever). Otherwise, you need to find your own way around the duplication issue (you asked for the recommended approach).

[Mirone]

Thanks for your clarification. So what I misunderstood is that YDoc not only store the document structure, but also store the editing history. That makes sense.

I think it’s better to let users to decide the sync timing and the storage structure. I’ll just provide some utilities for them.

[dmonad]

Great, I think that’s the way to go. Maybe I should explain that better in the docs.

It makes sense for authors of frameworks only to build the bindings to Yjs. The user then can choose existing backend-, and storage solutions from the Yjs ecosystem.

[Mirone]

Thanks for your patience. Great work! Will let you know once I finish my work in milkdown.