How to introduce YJs in our existing architecture?

1

Hey everyone!

I have been having a lot of fun trying our Yjs and I have successfully prototyped a lot of scenarios including ones with mongodb persistence.

However I am having a very hard time figuring out an architecture and plan to move our existing app to using yjs.

Our current frontend web app (React + Redux) connects to a backend express server which in turn connects to mongodb to make all sorts of CRUD operations.
Our data is stored in multiple collections as simple key-value docs on mongodb.

I am trying to understand what would be the easiest way to introduce Yjs. Ideally we want all clients to be able to collaborate and see each others updates in real time.

We have tried a few options and we havent had success. The biggest blocker has been our need to keep storing our data in mongodb as is. Our APIs and database are used in third party apps and we cannot remove them.

We keep running into problems around these:

  1. How to initialize the Y.Doc data in this scenario? Would the provider directly connect to the API or would a client on the frontend initialize the data after getting it from REST API.
  2. How to keep the data in DB (in normal key-value doc) updated along with Yjs

Any help and/or suggestions in architecture are highly appreciated!

Thank you very much
Vivek

2

The most important thing is that YJS has to be the source of truth. The current data that you have stored in MongoDB is just the most recent state. It does not have the complete history of edits, and without the history of edits the CRDT cannot merge concurrent edits. So you can’t initialize a Y.Doc from the latest state, but you can initialize the latest state from a Y.Doc.

I imagine you can maintain the MongoDB database in parallel by updating it in response to YJS updates. So something like this:

// load and bind the Y.Doc when the user is connected
const doc = persistence.getYDoc(documentName)
doc.on('update', () => {
  // get the current state
  // (or get specific values with ymap.get)
  const state = doc.toJSON()

  // update the db
  // (you may need multiple update calls with specific 
  // conditions to fit it into your existing schema.)
  db.collection.updateOne(/* query */, state)
})

If your existing API is read-only, it should continue working as-is as long as the MongoDB schema is preserved. If the API has write access, you’ll need to apply the changes to the server Y.Doc instead of directly to MongoDB. That way the Doc will update, conflicts will be handled, and the latest state will be propagated to MongoDB.

I’m just theorizing though… by all means if someone else has done this and has a better recommendation please chime in :).