Y-websocket debounce broadcast and merge updates

1

I’m using y-websocket to send updates to the back-end. It sends updates to the back-end on every change immediately.
Instead of reacting to every change, I’d like to be able to debounce that and accumulate the changes I need to send.
I’ve managed to achieve debouncing (using my own implementation of y-websocket), but I’m failing at aggregating many update messages (Uinit8Arrays) into one update. I’ve tried using Y.mergeUpdates and Y.mergeUpdatesV2, but the backend ws server is unable to process that message.
Any hints on how to merge these updates that come from the update event handler?

2

Hi @pgte,

How did you adapt the y-websocket code?

Y.mergeUpdates works on updates, not on messages generated by y-protocols.

const updates = []
ydoc.on('update', update => {
  if (origin !== this) {
    updates.push(update)
  }
})

// this should be triggered after a debounce
const sendCachedMessages = () => {
  const encoder = encoding.createEncoder()
  encoding.writeVarUint(encoder, messageSync)
  syncProtocol.writeUpdate(encoder, Y.mergeUpdates(updates))
  broadcastMessage(this, encoding.toUint8Array(encoder))
}
4 Likes
3

@dmonad Thank you, this is exactly the solution I was looking for!

1 Like
4

What is messageSync in this case?

5

It’s the bit used in the encoded message to indicate the type of message (sync)

6

Ahh, thanks a lot. This is required when you are sending messages to the y-websocket server, if I’m planning to caches the changes in the server I don’t need to encode right?

7

These message types are only used in the y-websocket communication protocol, so you don’t need to store or cache them.

Doc changes are encapsulated in Uint8Array updates.

8

Got it, I’m using y-websocket provider and receiving the updates on the server. Most of the time the updates works great. However sometimes I receive empty updates even I have not done anything.

E.g here is an decoded empty update size of 26b

{
  "structs": [
    {
      "id": {
        "client": 1847393735,
        "clock": 0
      },
      "length": 1,
      "origin": {
        "client": 1219761938,
        "clock": 0
      },
      "left": null,
      "right": null,
      "rightOrigin": null,
      "parent": null,
      "parentSub": null,
      "redone": null,
      "content": {
        "arr": [
          true
        ]
      },
      "info": 2
    }
  ],
  "ds": {
    "clients": {}
  }
}

how do I read this and stop them or ignore them.

9

That might be an awareness update, but I’m not really sure. You might want to check the message type after it has been decoded.

You could also store empty updates and compact them when they get too big. That strategy is used in many of the providers.

10

Also any pointers on how to check the message type from a decoded update?

11 
import * as decoding from 'lib0/decoding'
const buffer = new Uint8Array(e.data)
const decoder = decoding.createDecoder(buffer)
const messageType = decoding.readVarUint(decoder)
12

Oh, thanks a lot. Super helpful.

so i presume there are the message types and only Sync (0) has the data updates, and all the other types can be ignored is that correct?

image

13

Yes, only Sync messages contain update data.