`syncProtocol.writeUpdate` calls in `updateHandler` functions

1

I have a question about the updateHandler functions that call systemProtocol.writeUpdate

I found two instances where this happens (sorry the platform won’t allow me more than 2 links):

  • yjs/y-websocket/blob/master/src/y-websocket.js#L252
  • yjs/y-websocket/blob/master/bin/utils.js#L82

In both cases, before the syncProtocol.writeUpdate(encoder, update) call, encoding.writeVarUint(encoder, messageSync) call happens. Which means messageSync variable with value 0 is appended to the encoder.

This is the implementation of the writeUpdate function. It also appends a varUint to the encoder, but in this case it’s messageYjsUpdate with the value 2

So in the end, we have an array with contents: [0, 2, ...actualUpdateData].

As I understand readSyncMessage function expects a message with [messageType : varUint, ...updateData: Uint8Array] structure, so that second element, 2 will be treated as a part of the data.

Is this a bug or am I missing something?

Edit: Simplified/inlined code of what seems to be happening:

const update = new Uint8Array([...someStateUpdate])
const encoder = encoding.createEncoder()
encoding.writeVarUint(encoder, 0)
encoding.writeVarUint(encoder, 2)
encoding.writeVarUint8Array(encoder, update)
broadCastMessage(encoding.toUint8Array(encoder))
2

Hi @notgiorgi,

The y-protocols package really is a bit complicated. It certainly is a bit overoptimized and I never ended up writing good documentation for y-protocols.

y-websocket uses three protocols awareness, sync, and auth (although auth isn’t really being used).

y-protocols implements different protocols (all of the above). The sync protocol, for example, defines that when you receive a state-vector (syncStep1), you should reply with an encoded update syncStep2.

These are just common algorithms that I use in all the providers, so it made sense to me to outsource them to a separate package. Each protocol is optional, and you can completely reimplement them.

So here is how it works:
y-websocket receives a message from another peer.
If the first VarUInt is messageSync = 0, then forward the rest of the message to y-protocols/sync.
If the first VarUInt is messageAwareness = 1, then forward the rest of the. message to y-protocols/awareness.
And so forth…

In the case of y-protocols/sync:
If the first read VarUint is 0, then it is a syncStep1. Compute syncStep2 and reply to the sender.
If the first read VarUint is 1, then it is a syncStep2. Apply the upate, and fire the "synced" event.
If the first read VarUint is 2, then it is jus an update message. Apply it to the document.

encoding.writeVarUint(encoder, 0) // This message should be handled by the sync-protocol
encoding.writeVarUint(encoder, 2) // sync-protocol: it's an update message. Apply Y.applyUpdate(decoding.readVarUint8(decoder)) when received
encoding.writeVarUint8Array(encoder, update)

I hope this gave you a small overview of how y-protocols is being used. I think it makes sense to wrap the messages from y-protocols another encoder (e.g. protobuf) that is better documented. Since I want to minimize dependencies, I just ended up using lib0/encoding which is used by Yjs anyway.

3 Likes