Y.applyUpdate(doc, new Uint8Array(update)) is not updating the existing Ydoc consistently,

1

Hey Guys,
I am building a collaborative editor for my canvas editor, I am using Yjs for the collaborative editor. For every change one client is doing i am receiving that update at the backend and updating my yjs doc at the backend. I am using React and Nodejs along with sockets.
But one problem I am facing with this backend code (given), my backend Yjs doc is not updating and it’s saving the older data to Redis.

I tried this [Y.applyUpdate(doc, new Uint8Array(update)); & const diff = Y.diffUpdate(new Uint8Array(update), Y.encodeStateVector(doc));] but some times my older data is not updating : I am adding both backend and frontend code.

Canvas is JSON objects of elements
My data is the json object.

Any help will be grateful.

Yjs version Node : “yjs”: “^13.6.18”
Yjs version React : “yjs”: “13.0.0-103”

// Frontend React Code
const ymap = doc.getMap(‘json’);
ymap.set(‘data’, designer.canvas);
const update = Y.encodeStateAsUpdate(doc);
if (socket) socket.emit(‘update’, { update, id });

// Backend Node code

socket.on(‘update’, async ({update, id}) => {
doc.transact(() => {
Y.applyUpdate(doc, new Uint8Array(update));
});

let updatedJson = ymap.get(‘data’);
}

2

I understand that you maintain a Yjs document on the server and that you distribute messages via y-redis.

I suggest that you distribute the messages that you receive via the ydoc.on('update', update => {..}) event.

Alternatively, you can simply put all updates that you receive on redis.

Y.diffUpdate computes the diff between a Yjs document and a state vector. From the docs:

Y.diffUpdate(update: Uint8Array, stateVector: Uint8Array): Uint8Array Encode the missing differences to another update message. This function works similarly to Y.encodeStateAsUpdate(ydoc, stateVector) but works on updates instead.

It works similarly to Y.encodeStateAsUpdate(ydoc, stateVector).

That version is really old. I suggest that you update that ^^

3

Hey
Just to give you more brief idea about it, i am adding the detailed code and dummy json data which i want to store and provide update.

My json contains some key value pairs along with an array, sometimes it’s bigger in size

Here is the sample : `{
“content”: {
“result”: 1,
“lecture”: {
“uid”: “a1b2c3d4-e5f6-7890-abcd-1234567890ab”,

  "segments": [
    {
      "uid": "f1e2d3c4-b5a6-7890-cdef-456789abcdef",
     
      "canvas": {
        "uid": "canvas-wrapper",
        
        "elements": [
          {
            "uid": "elem001",
            "htmlContent": "<p style=\"text-align: left; font-family: Poppins; font-size: 92px; color: rgb(112, 112, 112);\"><strong style=\"font-size: 92px; font-family: Poppins; color: rgb(112, 112, 112);\"><span class=\"ql-cursor\"></span></strong></p>",
            "alignment": "left",
            "currentEditor": null,
              "start": 0,
              "end": 0
            },
            "originalSelectionRange": {
              "start": 0,
              "end": 0
            },
            "elementType": "textbox",
            "isFullySelected": false,
            "cornerRadius": 0,
            "spacing": 0
          }
        ],
        "activeElement": {
          "uid": "elem002",
          "shape": "curved-rectangle",
          "xCoord": 678.6,
   
          "elementType": "shapes"
        },
        "selectedElements": [],
        "blurredBackground": false
      },
      "serializedCanvas": "...",
      "thumbnailURL": "",
      "voiceover": [
        {
          "associatedElementId": "f1e2d3c4-b5a6-7890-cdef-456789abcdef",
          "voiceoverType": 0,
          "associatedElementType": 0,
          "order": 0,
          "voiceoverSegments": [
            {
              "uid": "vo001",
           
              "audioMetadata": null
            }
          ]
        }
      ],
      "order": 0,
      "avatarSettings": {
        "scaleHorizontal": 0.3,
      
        "isVisible": true
      }
    }
  ]
},

}
}`

Step 01 : I have this code at the frontend which is passing update to the backend,

FrontendCode :

const ymap = doc.getMap(‘json’);
ymap.set(‘data’, designer.presentation);
const update = Y.encodeStateAsUpdate(doc);
if (socket)
socket.emit(‘update’, { update, presentationId: presentationid });

Step 02 : Below is the backend code how i am saving data to redis and emitting the updates

Backend code:
socket.on(‘update’, async ({update, presencontentIdtationId}) => {
try {
const doc = getOrCreateDoc(contentId);
doc.transact(() => {
Y.applyUpdate(doc, new Uint8Array(update));
});
const diff = Y.diffUpdate(new Uint8Array(update), Y.encodeStateVector(doc));
Y.applyUpdate(doc, diff);
let updatedJson = ymap.get(‘data’);
let redis_key = updatedJson?.id;
let data = {
data: {
“status”: 1,
“content” : updatedJson,
“contentId”: updatedJson?.id,
}
}
await redis.set(redis_key, JSON.stringify(data), async err => {
if (err) {
console.error(‘Error storing JSON in Redis:’, err);
} else {
socket.to(contentId).emit(‘update’, update);
await redis.rpush(${redis_key}:deltas, Buffer.from(update).toString(‘base64’) );
}
});
} catch (err) {
console.error(‘Error handling update:’, err);
}
});

I am not using y-redis, please have a look of the above implementation and suggest something which can improve this. 

The problem is sometimes update is not applied to the ydoc, and it saves the older data into the redis, the updates are emitted but not saved in the db. 

 getOrCreateDoc(contentId); is giving me the existing doc present for this particular content.
 
 Please suggest something why this piece of code is not able to apply the difference : 
 
 doc.transact(() => {
      Y.applyUpdate(doc, new Uint8Array(update));
    });
    const diff = Y.diffUpdate(new Uint8Array(update), Y.encodeStateVector(doc));
    Y.applyUpdate(doc, diff);

It’s happening sometimes only. For each update, the applyUpdate is not functioning properly.

4

Hey , Can you please look into this ?

5

Hi I am eagerly waiting for the reply ? Can you please check it ?

6

Hi @shashank009 ,

This really is a lot of source code. I’m having trouble reading it. I understand this ticket is important to you, but I doubt that you will get the solution you are looking for like this.

Regarding this code… I’m really not sure what this is supposed to do.