Creating a custom provider

#1

In order to solve the problems I’m currently facing, I’ve started implementing a very first custom provider for Yjs, based on an example from the docs:

  class customProvider extends Observable {
    constructor (sharedDoc) {
      super()

      sharedDoc.on('update', (update, origin) => {
        if (origin !== this) {        // ignore updates applied by this provider
console.log('customProvider: update applied on sharedDoc',update)
          this.emit('update', [update])
        }
      })

    /**** listen to remote updates ****/

      this.on('update', (update) => {
console.log('customProvider: update received from remote',update,arguments)
        Y.applyUpdate(sharedDoc, update, this)   // "this" is transaction origin
      })
    }
  }

However, it seems that updates made to the sharedDoc from within my business logic will trigger both sharedDoc.on('update',...) and this.on('update',...) - with the latter applying the same update to the sharedDoc again.

Is this really the intended behaviour? Or, how can I avoid reapplying an already applied update within this.on('update',...)?

[Edit] simply adding this to this.emit('update', [update, this]) is not sufficient as the second argument will not be passed to this.on('update', (update, origin) => ...)

1 Like
#2

My current approach is to keep track of already applied updates internally:

  class customProvider extends Observable {
    /* private */ appliedUpdates = new Set()

    constructor (sharedDoc) {
      super()

      sharedDoc.on('update', (update, origin) => {
        if (origin !== this) {        // ignore updates applied by this provider
console.log('customProvider: update applied on sharedDoc',update)
          this.appliedUpdates.add(update)
          this.emit('update', [update])
        }
      })

    /**** listen to remote updates ****/

      this.on('update', (update) => {
        if (this.appliedUpdates.has(update)) {
console.log('customProvider: update ignored')
          this.appliedUpdates.delete(update)
        } else {
console.log('customProvider: update received from remote',update,arguments)
          Y.applyUpdate(sharedDoc, update, this) // "this" is transaction origin
        }
      })
    }
  }

Is there a better way? Why do I have to this.emit('update', [update]) anyway?

#3

According to my current knowledge, the following code is sufficient for the basis of a custom provider:

  class customProvider extends Observable {
    constructor (sharedDoc) {
      super()

      sharedDoc.on('update', (update, origin) => {
        if (origin !== this) {        // ignore updates applied by this provider
console.log('customProvider: update applied on sharedDoc',update)
// ...
        }
      })

      sharedDoc.on('destroy', () => {
console.log('customProvider: sharedDoc will be destroyed')
// ...
      })
    }
  }

(y-indexeddb also listens to sharedDoc.on('update',...) and sharedDoc.on('destroy',...) only and does not emit any 'update' events to itself)

Please, correct me if I’m wrong - and tell me why

#4

By the way, Y.logUpdate(Uint8Array) seems to analyze a given update and log any changes that update represents.

While I haven’t looked into that method with detail, it still looks promising and may help understanding Yjs’ update mechanisms.

#5

I haven’t made a custom provider before, but I am interested in what you come up with. It’s a shame the documentation on this was never completed.

Not sure why you’re subscribing to your own update event instead of just handling it in sharedDoc.on('update')? Maybe I am missing something though.

It may be the case that applyUpdate is idempotent, i.e. already applied updates will be ignored.

I’ve seen this bindState-style function used in various providers before. I think this should form the basis of a custom provider. It syncs the initial state and then watches for updates.

const bindState = async ({ roomName, doc }) => {
  const docPersisted = await db.getSharedDoc(roomName)
  const updates = Y.encodeStateAsUpdate(doc)
  await db.storeUpdate(docName, updates)
  Y.applyUpdate(doc, Y.encodeStateAsUpdate(docPersisted))
  doc.on('update', update => {
    db.storeUpdate(docName, update)
  })
}
#6

Thank you very much for your effort!

this.on('update',...) came from an example found in the docs - but it should be nonsense (and, after more investigation, I’m quite sure that it is indeed nonsense)

Your bindState looks indeed like a sketch for a custom provider although I would probably put the statements in a different order.

My current knowledge has been put into y-localstorage, together with the additional events I need

2 Likes