How do I merge a raw string to a document without duplicating?

1

Hello! I am a backend developer trying to venture into y.js.

I have built an architecture where a user may edit a description field from a text editor (which will support live editing through y.js), but also directly from the API using a raw string.

What then happens is that user A updates the description through the API directly on the database, and user B then connects to y.js, which has its own cache/storage storing a completely different data.

I may be approaching this architecture wrong, but I was wondering if there is a way of getting this data from the API, convert it into a y.js document and attempt to merge with the document from y.js.

I don’t see any alternative since we technically have 2 sources of truth.

2

The binary YJS data has to be the source of truth in order to get conflict-free replication. It’s not possible to convert static data into a YJS Doc, since static data does not have the history of edits that are needed to resolve conflicts from offline changes.

Your API could load the YJS Doc, modify it, propagate the update to all clients, and persist the new state to the database.

1 Like
3

Hey, jumping in here with a similar scenario, hope that’s cool.

I have a productivity app that is powered by Dexie Cloud. In this app I have multiple projects, tasks, lists and more. Data I stored with Dexie.js and synced through their Cloud Service. The app is offline first and all changes are saved into indexeddb first.

In every task and projekt the user is able to edit a note that is a TipTap-editor powered by y.js.

It’s here my “issue” is introduced. All the Dexie data is synced and stored in the local indexeddb but the y.js data is not. y.js data is only stored if accessed on this device first. There are some scenarios where a task is created, all data synced and available but not yet opened on a specific device and therefore does not have a doc to work with. The first option: Populate the editor with the stored data (YDoc stored in Dexie and saved by the latest edit). I have tried this but then the data is duplicated which is not desirable
Second option: Give the user a blank editor that later is merged when online

But is there a third option here?

Can I store the lates YJS Doc with history in the task and on these scenarios inject a local version based on the latest edited and synced version stored in my object? Similar to the IndexeddbPersistence.

If so I could populate the editor with the stored data and the history and every update done offline after this init will be handled the same way the rest is.

4

I was wondering if there is a way of getting this data from the API, convert it into a y.js document and attempt to merge with the document from y.js.

A Yjs document is like a git repository with a change history. Yjs itself doesn’t calculate a diff with remote versions of the document. It’s really just a change history. So no, that’s not possible…

I don’t see any alternative since we technically have 2 sources of truth.

This seems to be a common denominator between these questions.

You can’t really have two sources of truth unless you can devise an algorithm to sync them (it’s not that trivial). Similarly, there is no viable algorithm that syncs a git repository with a mercurial repository (I mean, many have tried, but it never worked out).

My suggestion is that you use Yjs as the source of truth and reflect the changes to the database.

Anything else will likely be buggy. Give me an algorithm, and I’ll break it.

In every task and projekt the user is able to edit a note that is a TipTap-editor powered by y.js.
All the Dexie data is synced and stored in the local indexeddb but the y.js data is not. y.js data is only stored if accessed on this device first.

What happens when two users open the same task at the same time? It will likely duplicate as well, because Yjs doesn’t merge, it just manages a change history.

Second option: Give the user a blank editor that later is merged when online

That works, but it would be sad if your app wouldn’t work offline anymore.

My suggestion is to store Yjs docs directly in Dexie. That is certainly possible. Maybe have a look at y-indexeddb and the document-updates section in the docs