I’ve successfully integrated YJS with TipTap using a custom provider, and it’s functioning smoothly. Now I need to save additional data pertaining to graphics, which is in the form of complex, deeply nested JSON objects.
I looked at the documentation and saw the Y.Mapset method, but it doesn’t accept JSON objects as value directly. Should I create nested maps and arrays recursively using the Y.Map and Y.Array types?
If so, how can I get the entire json object back while receiving an update?
For clarity, here’s an example of structure I’m dealing with:
However, if someone looks at this topic and knows how JSON works with YJS: how does it manage updates inside the JSON? For example if I have a very large JSON object and I update only one value, what will the library send as an update?
If I use Y.Map and Y.Array, would it work differently?
I went to test this out myself, and I’m quite surprised with the results.
If you mutate an object that has been added to a Y.Map, it will modify the object that map.get returns, but no update will be created. Thus the shared type is not really modified, and certainly not synced with other providers. The shared type is only updated when map.set or map.delete is called. As you can see in the example below, no update is created when the deep property is modified.
As deceptive as this is, I assume that YJS does this to avoid doing a deep clone. The documentation does not contain any prescriptions against mutating objects held in a shared type, but I would say that is an important invariant.
The correct approach is to call map.set after you mutate an existing object. That will generate a new update.
Thank you for trying out and sharing this result. This is good to know!
I’m actually using a .set() while updating the value, but I wondered if there is a way to optimize the structure to get small updates while updating values. For example, if I have a large graphical element as a json and I update its position (nested value in the json), is there a way to only get this change as an update instead of the entire json object being sent in each update?
import * as Y from "yjs";
const doc = new Y.Doc();
doc.on("update", (update) => {
console.log("update created", update.length);
});
const map = doc.getMap();
const obj = {
a: {
b: {
c: 0,
d: "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.",
e: "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place."
},
},
};
map.set("obj", obj);
// update created (769)
obj.a.b.c = 1;
map.set("obj", obj);
// update created (777)
obj.a.b.c = 2;
map.set("obj", obj);
// update created (777)
In this example, the entire object is sent every time the number changes.
Ok, got it, thanks for the confirmation, that’s what I was assuming.
I just tried and it only works if I manually update the values through the Y.Map, not if I recreate the Y.Map. Unfortunately in my case the object comes already updated, so I would have to compare each nested value and reflect the changes using the yjs setter methods.
I’m assuming other developers are directly the Y types in their apps instead of using objects?
import * as Y from "yjs";
const doc = new Y.Doc();
doc.on("update", (update) => {
console.log("update created", update.length);
});
// -----------------------------
// Using pure JS objects
// -----------------------------
const map = doc.getMap();
const obj = {
a: {
b: {
c: 0,
d: "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.",
e: "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place."
},
},
};
map.set("obj", obj);
// update created (769)
obj.a.b.c = 1;
map.set("obj", obj);
// update created (777)
obj.a.b.c = 2;
map.set("obj", obj);
// update created (777)
// -----------------------------
// Using Yjs types
// -----------------------------
// Recursively traverse the object and convert it to a Yjs object
function encode(jsObject) {
if (typeof jsObject === 'object') {
const yObject = new Y.Map();
for (const key in jsObject) {
yObject.set(key, encode(jsObject[key]));
}
return yObject;
} else if (Array.isArray(jsObject)) {
const yArray = new Y.Array();
for (const value of jsObject) {
yArray.push([encode(value)]);
}
return yArray;
} else {
return jsObject;
}
}
const yMap = doc.getMap();
yMap.set("yObject", encode(obj));
// update created (812)
obj.a.b.c = 3;
yMap.set("yObject", encode(obj));
// update created (816)
yMap.get("yObject").get("a").get("b").set("c", 4);
// update created (27)
I had a similar case. I was adding YJS to an existing Redux application. In some cases I captured individual deltas and propagated them through a separate channel apart from the Redux state so they could be applied ad hoc to the shared types in some middleware. In other cases I actually diff the objects to generate granular updates. Either of those approaches can be effective, although obviously YJS strongly favors shared types as both the source of truth and the source of change.
This technique works well, but I just want to give a warning that it must be done synchronously, otherwise it can delete unexpected keys. Specifically, if an update comes in concurrently from another client, it may not be integrated into the object in time for the user’s edit. Then syncJsObjectToYMap will think that one of the new keys should be deleted. In this case it’s impossible to tell the difference between a local deletion and a remote addition if there is any asynchrony between the edit and the update.
Ensuring that syncJsObjectToYMap is called synchronously with the event handler should prevent this from happening (I hope).
I’m also looking into syncing Y.maps to my own class objects (which also have validators and should act as a source of truth). I was wondering @erwan if you had any more experiences on this, and if you chose one of the above packages?
On a sidenote, would Y.map be able to sync class instances? For example:
const doc = new Y.Doc();
class TaskManager {
constructor (yDoc) {
this.id = getUUID()
this.tasks = ydoc.getArray('taskslist_'+this.id)
}
addTask() {
// validate input
this.tasks.push( new Task() )
}
}
Shared types can only store serializable data, so that means no class instances or functions. (The same restriction exists for pretty much any database though.)
You can always convert between class instances and YJS shared types, say with the help of a serialize instance method and deserialize static method. It’s an extra layer of abstraction, but if you want a custom OO interface that hides the YJS shared types, you could do it.
That’s why I said that’s it a “naive” implementation (more like a proof of concept).