Hello! We’re developing a collaborative vector drawing application and had some questions about our usage of YJS, specifically what we can do to
mergeUpdates to make it block less.
On the server our usage of yjs is somewhat similar to the ’ Syncing clients without loading the Y.Doc’ example in the readme, where our server both broadcasts updates to all the clients, but also uses
mergeUpdates to keep a running snapshot of the entire document, which is then stored. If that’s not intended/recommended, let us know!
On the client we’re naively/recursively converting our data structure into a Y.Doc, which for more complicated canvases with more complicated shapes (specifically text) end up with both a lot more data (10s of MB), and a lot more structs (
yjs.decodeUpdate(...).structs.length gets to be about few million after we merge down the updates).
We’re using Y-Websockets to transmit this data to our backend, which then uses a pub-sub system to keep all clients in sync. We’ve found with the size of the updates we’re handling,
mergeUpdates is becoming a problem.
Running some benchmarks with updates from an example canvas, where I duplicated all elements in the canvas a bunch of times, resulted in the following: yjs-mergeUpdates-example.ts · GitHub.
~1.3 seconds doesn’t seem like a lot on the surface, but it’s not the largest example we could generate, and that time is all spent synchronously merging that update, meaning nothing else can happen (like distributing new updates) until it’s done. It also uses a decent amount of memory (and obviously more if we make the updates even larger).
In order to keep updates flowing, I had the process that distributes updates not do the merge, but just store the update on a queue of updates to be merged, and then had a queue worker actually do the merging. This is still having issues when the merges take long enough our health endpoint (as these are running in a cloud environment) doesn’t always respond in time.
We’ve considered a few ways forward:
Have fewer structs by serializing updates to our datastructure past some depth. Since right now we recursively convert everything to YJS types, every number of every part of every path becomes its own struct in the final YJS doc (at least in my understanding). If instead we pick a level of non-mergability, like a path in our vector drawing, and never recurse deeper than that, we should have fewer structs to merge, and so maybe
mergeUpdatescould be much faster? Basically we don’t know if having millions of structs is reasonable / should be addressed.
Find a way to break a single mergeUpdates call into an async operation that can yield to other processes. We saw someone else asking about something similar-ish (Split update into smaller updates), and I can see why it’s not recommended, as to use the LazyStructReader/Writer it seems like we’d have to either fork yjs, or at least copy in a lot of logic from mergeUpdates to add some
setImmediatecalls and a Promise wrapper.
Throw mergeUpdates into a nodejs worker thread, which should be able to pass references to Uint8Arrays (i.e. without copying), and since it’s actually another thread, should not block responding to health checks.
Make sure no updates from clients are larger than some size. Right now every user modification to the canvas can cause a number of updates, which we
yjs.transacttogether. If instead we batched updates less to make sure they’re no larger than some size, maybe
mergeUpdatescould be a lot faster?
Curious of people’s thoughts on it, thanks!