How to label a transaction and add event metadata?

1

Let’ say user does an action called ‘turn on share’
It triggers the following transaction

const toggleSharing = () => {
    yDoc.transact(() => {
      yDoc.getMap().set("sharing", !doc().sharing)
      if (doc().sharing && !doc().roomId) {
        yDoc.getMap().set("roomId", crypto.randomUUID())
      }
    })
  }

And I listen to local changes in “update” handler and send the message to other peers

yDoc.on("update", (update, _, __, tr) => {
    console.log('update', tr.local, tr.origin)
    if (tr.local) {
      // change made by current user, send to peers
      const base64Update = fromUint8ArrayToBase64(update)
      const message = {type: 'docUpdate', data: base64Update}
      sendMessage(message)
    }
    // update local doc
    setDoc(yDoc.getMap().toJSON() as any)
  })

And on peer side I do

const {type, data} = message
switch (type) {
      case "docUpdate": {
        Y.applyUpdate(yDoc, fromBase64ToUint8Array(data))
        break
      }

The underlying data gets synced this way, but the peer doesn’t know what “action” caused the data to change.

Is there any way to pass event data with customer properties as part of the transaction?
From the docs I can see that transaction takes in an origin property of any type so I thought i could potentioally do

 const toggleSharing = () => {
    yDoc.transact(() => {
      yDoc.getMap().set("sharing", !doc().sharing)
      if (doc().sharing && !doc().roomId) {
        yDoc.getMap().set("roomId", crypto.randomUUID())
      }
    }, {author: user.id || 'server', event: 'toggle on sharing', ...anyOtherData})
  }

And in the update listener

yDoc.on("update", (update, _, __, tr) => {
    console.log('update', tr.local, tr.origin)
    const {author, event, ...props} = tr.origin
    if (tr.local) {
      // change made by current user, send to peers
      const base64Update = fromUint8ArrayToBase64(update)
      const message = {type: 'docUpdate', data: base64Update, event, ...props}
      sendMessage(message)
    }
    // update local doc
    setDoc(yDoc.getMap().toJSON() as any)
  })

And on the peer side

const {type, data, event, otherProps} = message
 switch (type) {
      case "docUpdate": {
        Y.applyUpdate(yDoc, fromBase64ToUint8Array(data))
        if (event === 'toggle on sharing') {
           // Do something, like show a toast, etc
        }
        break
      }

Is this way of passing metadata as part of origin ok or is there another recommended way of doing so?

2

You probably worked with event-based systems before. It is a valid approach to store sequences of events and replay them on remote clients to achieve convergence (everyone ends up with the same representation of data).

However that’s not what Yjs is. There is no guarantee that “events” will be replayed in the same order. Once the data is merged, there is no way to recover any order of transactions.

Yjs syncs state, not event orders. Hence, Yjs only gives you events for the data that changed.

3

Thanks for the quick reply.
I actually don’t require replaying of events
My main intention here was to run some side-affects (either on client or the server) when certain keys in yDoc changes

One way I was considering was to check if the data point changed in the “update” event handler (by comparing with the past value) and trigger side-affects. But this was too cumbersome to setup, so I thought instead if I just label the updates, I can execute side-affects based on the labels which i think is easier to set up
Once the session ends, I don’t really care about the labels anymore just the yDoc state

4

Also I’m struggling with how to maintain the integrity of the document state in a collaborative environment.

I have a Y.Doc() structure that looks like this:

const doc = new Y.Doc();
doc.getMap().set('language', 'python');
doc.getMap().set('executionStarted', false);

On the UI, I have two actions:

  1. Change the editor language.
  2. Execute the currently selected language’s code on the server.

Constraint: Once the execution has started, the language cannot be changed until the execution is complete.

Consider the following scenario:

  • User 1 triggers code execution while the language is set to ‘python’.
  • At the same time, User 2 attempts to change the language to ‘js’.

If both updates get applied, the UI will show ‘js’ as the current language but execute the code in ‘python’, which is not what is desired.

What is the recommended way to maintain document integrity in such collaborative environments, ensuring that conflicting updates like these do not lead to inconsistent states?

5

For the first question: I think the observer API might be more suitable than listen to updates: https://docs.yjs.dev/api/shared-types/y.map#observing-changes-y.mapevent

Regarding the other question:

It’s really up to you to model the data so this can’t happen. Usually, I suggest to keep related information together. Not everything must be a Y.Map. You could simply replace information to keep it consistent.

So instead of …

const doc = new Y.Doc();
doc.getMap().set('language', 'python');
doc.getMap().set('executionStarted', false);

You do

const doc = new Y.Doc();
doc.getMap().set('runtime', { language: 'python', executionStarted: false })

and replace the whole JSON object when something changes. This is what the did in the Jupyter project (see cell output in YModel), which has similar requirements.

Alternatively, you can make the JSON object a Y.Map, but replace the whole Y.Map whenever you change the language.

6

Thanks,
What you suggested makes sense. Will try it out and see how it works in practice

Sry, but 1 more doubt, if I do this

Alternatively, you can make the JSON object a Y.Map, but replace the whole Y.Map whenever you change the language.

Is there any performance downsides of creating new Y.map?
Ex which one would be better (in terms of performance)

const newMap = new Y.map()
newMap.set("prop1", val1)  // all other props are undefined
yDoc().getMap().set("console", newMap)

Or

const x = yDoc().getMap().get("console")
x.set("prop1", val1)
x.set("prop2", undefined)
x.set("prop3", undefined)

…

1st way looks easier to write/maintain but is there any performace downside in creating a new map everytime rather than mutating the same one?

7

A Y.Map allows you to granularly change its properties. A JSON object can only be replaced.

The general rule is to create only as much granularity as necessary - not more.

Less granularity creates less operations in Yjs and is hence more performant.

Replacing content will allow Yjs to efficiently garbage-collect all of its children. So replacing a whole Y.Map with almost identical values doesn’t hurt performance.

Regarding the second example: I don’t suggest to ever set undefined to indicate that there might be content.

–

the first example would be preferable. However replacing content using JSON instead of a Y.Map is actually more performant as it creates fewer Yjs types (the only cost is the loss of granularity that you might not need).

1 Like