Encode state as update and self state vector return big updates

1

Hi, all, I’m new to yjs. I use encodeStateAsUpdateV2 to get the minimized updates on the server side.
When I call encodeStatesAsUpdateV2 use the current state vector. it returns big updates, almost full updates( closed to not pass sv). According to the API document, only write the missing differences to the update message.

Is there something wrong with my way of coding, or my understanding of the API?

const updateNeedApply = readFileSync('./tmp/1412294364815704065_1665017489422151681_current')

const ydoc = new Y.Doc({
})

Y.applyUpdateV2(ydoc, updateNeedApply)
const sv = Y.encodeStateVector(ydoc)
const updates = Y.encodeStateAsUpdateV2(ydoc, sv)

console.log(`sv from update: ${Y.encodeStateVectorFromUpdateV2(updates)}`) // 0
console.log(`updates len: ${updates.length}`) // 7970726
2

I haven’t worked with the Updates API directly myself, but I am seeing that you are using the state vector on the same Doc it was generated from. I think it only works when using it on a different Doc (i.e. to synchronize them).

In the documentation, under Example: Sync two clients by computing the differences, notice how diff1 is using stateVector2, and diff2 is using stateVector1.

3
4

@jarone @raine

Thank your reply!

Actually, I changed my test code to the below, it has the same result.

const updateNeedApply = readFileSync('./tmp/1412294364815704065_1665017489422151681_current')

const ydoc1 = new Y.Doc({})
const ydoc2 = new Y.Doc({})

Y.applyUpdateV2(ydoc1, updateNeedApply)
Y.applyUpdateV2(ydoc2, updateNeedApply)

const sv1 = Y.encodeStateVector(ydoc1)
const sv2 = Y.encodeStateVector(ydoc2)

console.log(`sv1 == sv2: ${sv1.toString()==sv2.toString()}`) // sv1 == sv2: true

const diff1 = Y.encodeStateAsUpdateV2(ydoc1, sv2) 
const diff2 = Y.encodeStateAsUpdateV2(ydoc2, sv1)

console.log(`diff1: ${Y.encodeStateVectorFromUpdateV2(diff1)}  len: ${diff1.length}`) // diff1: 0  len: 7970726
console.log(`diff2: ${Y.encodeStateVectorFromUpdateV2(diff2)}  len: ${diff2.length}`) // diff2: 0  len: 7970726

When I call parseUpdateMetaV2 to the diff1 and diff2 updates, its return

{"from":{},"to":{}}

According to the API document, should the diff be very tiny? now the size of the update is 7M when the size of the updateNeedApply update is 10M. :sob:

5

hi, @baolin

Hope this demo can help you:

const Y = require('yjs')

const guid = 'id'

const ydoc1 = new Y.Doc({ guid })
const ydoc2 = new Y.Doc({ guid })

const ytext1 = ydoc1.getText('text')
const ytext2 = ydoc2.getText('text')

ytext1.insert(0, 'a')
ytext2.insert(0, 'b')

const stateVector1 = Y.encodeStateVector(ydoc1)
const stateVector2 = Y.encodeStateVector(ydoc2)

const diff1 = Y.encodeStateAsUpdate(ydoc1, stateVector2)
const diff2 = Y.encodeStateAsUpdate(ydoc2, stateVector1)

Y.applyUpdate(ydoc1, diff2)
Y.applyUpdate(ydoc2, diff1)

console.log({
  doc1: ydoc1.toJSON(),
  doc2: ydoc2.toJSON(),
})

// { doc1: { text: 'ba' }, doc2: { text: 'ba' } }
2 Likes
6

Hi, thanks for your explanation.

After reading and debugging the applyUpdateV2 function’s source code, I found some missing updates in the PendingStruct or some pendingxxx else. That is why generating updates from sv was so huge. It’s my fault.

Appreciate your help again!