Issue related to rendering snapshot version changes from database

1

I am reaching out regarding an implementation challenge I am facing while integrating ProseMirror version history using the Yjs-demo code available here (https://github.com/yjs/yjs-demos/tree/main/prosemirror-versions).

To provide some context, I am storing base64-encoded snapshots on the server. When attempting to retrieve and decode these snapshots upon a page refresh, I encounter an issue – the ProseMirror editor does not render the expected diff changes. Strangely, I can observe the snapshot version changes successfully when staying on the page without a refresh.

I am seeking your expertise to identify and address what might be going wrong in this scenario. Specifically, I have ensured the proper initialization of both the Y.Doc and ProseMirror editor to match the state of the snapshot. Additionally, the initial state of the ProseMirror editor is set correctly.

If you could spare a moment to review the provided information and offer insights into potential solutions or debugging approaches, it would be greatly appreciated.

@raine @dmonad Appreciate your help…

Implentation Code:

const versions = ydoc.getArray('versions')

 const addSnapshotToDatbase = doc => {
    const snapshot = Y.snapshot(doc)
    const encodedSnapshot = Y.encodeSnapshot(snapshot)
    // encode the snapshot and store in db
    const binaryString = String.fromCharCode(...encodedSnapshot);
    const base64Encoded = btoa(binaryString);

    // code to send the base64Encoded string to database  and store on server
}

const getSnapshotsFromDatabase = () => {
    const versionBlobFromDatabase = "SGVsbG8sIFdvcmxkIQ==";
    // decode encoded string
    const decodedString = atob(versionBlobFromDatabase);
    const decodedArray = new Uint8Array(decodedString.length);
    for (let i = 0; i < decodedString.length; i++) {
        decodedArray[i] = decodedString.charCodeAt(i);
    }
    
    renderVersion(editorview, decodedArray, null)
}

const renderVersion = (editorview, snapshotversion, prevSnapshot) => {
    editorview.dispatch(
        editorview.state.tr.setMeta(
            ySyncPluginKey,
            { 
                snapshot: Y.decodeSnapshot(snapshotversion),
                prevSnapshot: prevSnapshot == null ? Y.emptySnapshot : Y.decodeSnapshot(prevSnapshot)
            }
        )
    )
  }

<button onclick="addSnapshotToDatbase(ydoc)">Capture Snapshot</button>

<button onclick="addSnapshotToDatbase(ydoc)">View Snapshot Diff</button>
2

I’m sorry, I don’t have any experience with ProseMirror.

I would triple check your decoding algorithm. That’s the part that’s most opaque and error-prone.

3

@raine is there any way we can extract ydoc from Y.snaphot ?

4

No, a snapshot is just a single state of the data within a Doc. Docs contain the entire history of edits.

One thing I noticed is that the second argument of renderVersion expects a version object:

{
  date: number,
  snapshot: Uint8Array,
  clientID: string
}

But you are just passing Uint8Array (decodedArray).

5

decoded array is a Uint8Array… it’s a base64 decoded string

the name is confusing here… sorry about that…

6

I think the naming is fine. You have a type error though. You’re passing a Uint8Array where a { snapshot: Uint8Array } is expected.

7

I see what you meant… it’s a copy paste issue… I only provided main part of the code
but in my code I’m passing version object

edited my question

1 Like
8

I might try starting with a working example like the demo and closely comparing the two, or incrementally porting the demo over to your designed use case. There are too many potential causes now (decoding, snapshot handling, ProseMirror, …), so from a debugging perspective I would be testing different hypotheses and narrowing down the problem space.

Sorry I can’t be of more help.

9

it’s not working on yjs demo code as well

10

hi @raine, I’m working on the same project and we were able to narrow this down to interaction with y-websocket. If I write the document state and snapshots to localStorage instead, it works across refreshes. I created a branch of the prosemirror-versions demo to reproduce the issue and included a screencap: Show snapshots issue with y-websocket by rebolyte · Pull Request #61 · yjs/yjs-demos · GitHub

My understanding of this so far:

  1. Client creates a ydoc (gets a client ID, call it 123), establishes a connection to websocket server. Server creates a ydoc (gets a client ID, call it 456). Both client and server both send syncStep1 and respond with syncStep2. Initially, the document is empty on the client, so only change is from server.
  2. Client makes changes as ID 123. Client captures snapshots, which contain clock (sequence num) for IDs 123 and 456. Rendering the snapshots is done by y-prosemirror iterating through which items were added/removed here, which works since the current IDs match?
  3. User refreshes the page, so now client has new ID 789 and server has 012. Snapshots don’t line up, because changes made by ID 123 were now transacted on the document by 012?

This is all theory at this point. I don’t yet know enough of how the Snapshot deleteSet/state vector is applied to the internal representation to see the why here. If we got this demo working with y-websocket it would resolve our issue.

11

Also not sure why incrementing previous snapshot’s client ID is necessary here:

    prevSnapshot.sv.set(
      prevVersion.clientID,
      /** @type {number} */ (prevSnapshot.sv.get(prevVersion.clientID)) + 1
    );
12

Hi @raine any ideas here?

13

It’s strange that it works with indexeddb but not websocket. However that should give you a good basis to narrow down the problem because you have a good state and a bad state that you can triangulate from. Does the Doc get synced the same from the websocket as indexeddb? Are the arguments passed to renderSnapshot exactly the same between the websocket version as indexeddb?

I’ve never actually used ProseMirror myself, so I can’t offer anything more specific. I hope you can figure it out.