Are y-leveldb and y-indexeddb interoperable?

I’m new to Y.js and am running into a problem setting up a simple local-first architecture. I get an exception in the browser when it receives the first update from the server.

I have a simple server that persists documents in y-leveldb and some code that runs in a browser to persist data using y-indexeddb. I have a node client (not shown) that successfully interacts with the server, storing data in another y-leveldb and syncing with the server, so I know that this minimal server works as expected. The server is passive, not generating any changes to the data itself, all changes originate in the clients.

Here is the server code, minus some console logging, and broadcasting to other clients:

const persistence = new LeveldbPersistence('./db/server');
persistence.getYDoc('personal-notebook').then((doc) => { 

  const ydoc = doc;

  // Simple WebSocket server for collaboration
  const wss = new WebSocket.Server({ port: 8081 });
  wss.on('connection', (ws) => {
    ws.send(Y.encodeStateAsUpdate(ydoc));
    ws.on('message', (message) => {
      try {
        const msg = new Uint8Array(message);
        Y.applyUpdate(ydoc, msg);
        persistence.storeUpdate('personal-notebook', msg);
        console.log('Document state after update:', dump(ydoc));
      } catch (error) {
        console.error('Error processing message:', error);
      }
    });
  });
});

Here is the browser code:

import * as Y from 'yjs'
import * as IndexeddbPersistence from 'y-indexeddb'

const ydoc = new Y.Doc()
const indexeddbProvider = new IndexeddbPersistence.IndexeddbPersistence('personal-notebook', ydoc)
await indexeddbProvider.whenSynced // Ensure the provider is synced before proceeding

const ws = new WebSocket('ws://localhost:8081')
ws.onopen = () => {
  console.log('WebSocket connection established')

  ws.addEventListener('error', error => {
    console.error('WebSocket error:', error)
  })

  ws.onmessage = (event) => {
    try {
      const message = event.data;
      const msg = new Uint8Array(message);
      Y.applyUpdate(ydoc, msg);
    } catch (error) {
      console.error('Error processing message:', error);
    }
  };

  ydoc.transact(() => {
    const x = Y.encodeStateAsUpdate(ydoc);
    ws.send(x);
  }, 'browser')
}

Ther server sends its update to the browser and receives and processes the browser’s update just fine. The browser sends its update just fine, then throws an exception (Unexpected end of array)

The exception is thrown in yjs here:

const readClientsStructRefs = (decoder, doc) => {
  /**
   * @type {Map<number, { i: number, refs: Array<Item | GC> }>}
   */
  const clientRefs = create$5();
  const numOfStateUpdates = readVarUint(decoder.restDecoder);

This fails whether the server database is empty or not, in case that is significant.

Have I made an obvious mistake? I assumed that y-leveldb and y-indexeddb are interoperable, but now I want to check all my assumptions! :grinning_face_with_smiling_eyes:

They are completely interoperable. The issue is probably that there is existing content, and the sync isn’t working correctly. The error message suggests that the message got corrupted (or is only sent partially). I would check the following things:

  • do the messages get corrupted when sending? You could check the hash of the massages, or simply compare the lengths of the messages. This is the most likely case.
  • are you using the same Yjs release on backend and client (unlikely, but there might be incompatibilities - especially between Y-CRDT and Yjs).
  • Do the messages get corrupted in the database? Again, it would make sense to compare the hashes of the encoded documents after they are synced.