Local data lost with IndexedDB + Websocket providers

1

My application is using YJS with IndexedDB + Postgres (via websocket) persistence and I have a client that lost all of his data.

I figured out that the client data was never written to Postgres because it never closed the application - so the websocket connection did not close neither, and the data was not written to Postgres.

It may or may not be useful, but the client is using Safari.

I have more troubles to figure out how its data was not saved locally with the IndexedDB provider. I looked at the code of y-indexeddb and this is what I understand:

  1. whenever an update happens to a document, it is added to IndexedDB
  2. when the number of updates is greater than 500, all updates are merged into one big update

That seems pretty solid, and completely irrelevant to the fact that the client never closed the tab.

Could it happen that since the latest data has never been saved into Postgres, the Postgres data overwrote the local data when the client finally refreshed the page?

If a merge of all the updates then happened in IndexedDB, all previous data would be cleaned up even locally and could not be found again.

2

In general, IndexedDB should still work and persist even if other providers do not.

Any data initialization or setting defaults has the risk of overwriting data that is still loading. Nested Y.Map types can be at risk of being overwritten if an existing key is used and the nested map is unintentionally re-initialized.

(There’s also the fun fact that IndexedDB isn’t really persistent on mobile devices and is wiped periodically when the OS needs space or decides to have a paroxysm.)

2 Likes
3

Indexdb is indeed super fickle on mobile. Would it be a better idea to use the levels adapter if one has the option (e.g. for an electron app)? Is it more stable?

4

We’re basically stuck with IndexedDB on web. There’s SQLite in most browsers, but in Safari it has to be emulated. Native mobile environments seem to have more stable database systems.

However, this departs from the OP so I think further discussion should be in its own topic.