Decoding YJS messages

1

I’m trying to get started with Yjs, and I’d like to be able to see the format of the update messages to better understand what’s going on. Additionally, I’m interested in trying to persist Yjs data in a format which can be queried, so simply persisting the updates in binary will not work for my use-case.

Is there a public API for decoding Yjs messages into JSON? Or some other human-readable format. I tried looking through the code to figure it out, but got lost in a sea of decoders.

Thanks!

1 Like
2

Hi @danny-andrews,

Multiple people have asked for a feature to decode messages in a JSON format and I’d love to receive a PR that does such a thing.

But even with such a feature you probably won’t be able to persist Yjs in a queryable format (unless you reimplement Yjs to work directly on a database, which would require you to be very familiar with the inner workings of YATA & Yjs). Before you can read the document you must apply all updates that were ever created. This is just how Yjs, and similar CRDTs for linear structures work.

A better solution is to extract data from the Yjs document and store that in a queryable format into the database. For example, after each change (after a debounced timeout), you could load the Yjs document to memory, extract the data you are interested in, and store that in a database.

Alternatively, you could have a look at Riak or AntidoteDB. They use CRDTs to resolve conflicts directly on the database.

If you are interested in the inner workings of Yjs you could have a look at some internal information I published here: https://docs.yjs.dev/api/internals

3

Thanks for the reply. I was thinking of using a solution similar to what you suggested, i.e. storing the current state of the document in JSON alongside the encoded yjs document.

Anyway, I might take a stab at a PR in the next few weeks.

Cheers!

1 Like
4

Hi @danny-andrews, I also want to be able to decode updates to JSON. Are you still planning on doing this PR?

5

Hi @omerd, not at the moment. My OpenSource work is taking a back seat to looking for employment.

6

Hi @danny-andrews and @omerd, as part of another feature implementation, I implemented a basic logging utility for updates. https://docs.yjs.dev/api/document-updates

You can now perform. Y.logUpdate(update) to log the content of an update message to the console.

2 Likes
7

Radical! Thanks @dmonad.

8

Amazing!!! Thank you @dmonad!