How to store ydoc in a database?

1

Hello everyone! Im using yjs to to create a multiplayer code editor with y-codemirror and y-websocket as provider. I have a bytea field in my pg database where I want to store the entire ydoc for persistence reason. AFAIK I should do the following but not sure how.

  1. Y.encodeStateAsUpdate on the client to get the current state and pass it to the server.
  2. Save the incoming state in the database.
  3. when a new client joins they should get the state from the server.
  4. use Y.applyUpdate to populate the current state with the server.

Please guide me im really lost.

2

Hi, welcome.

If you’re looking to build a pg provider, your best bet is to model it after an existing database provider:

If you are already using y-websocket, you don’t need to worry about syncing with clients (most of your steps). Just let y-websocket sync all clients, and create a db provider on the server. If you connect y-websocket and your custom db provider to the same YJS Doc, it will persist the synced Doc to the db. The networking provider handles syncing between clients or between client and server, and the database provider handles persisting to the database.

You can also see how y-websocket uses its built-in leveldb persistence to persist Docs.

Unfortunately there is not yet an abstract db provider that would allow you to simply plug a YJS Doc into the bytea field in your db. I think this is much needed to make it easier for developers to use YJS with different databases.

1 Like
3

Thank you for your reply!

Im just a beginner and dont think I could build a provider for postgres storage. Before integrating yjs I was storing the editor contents by sending a POST req from client to server. I want to know if this is feasible in yjs eco system.

Here is my main problem

For example when I do Y.encodeStateAsUpdate(ydoc1) I get a UInt8Array with value { "0": 1, "1": 1, "2": 136, "3": 145, "4": 193, "5": 243, "6": 6, "7": 0, "8": 4, "9": 1, "10": 11, "11": 99, "12": 111, "13": 110, "14": 116, "15": 101, "16": 110, "17": 116, "18": 72, "19": 84, "20": 77, "21": 76, "22": 1, "23": 100, "24": 0 }

lets store it in foo and do Y.applyUpdate(ydoc2, foo) and boom I get an error.

Unhandled Runtime Error

Error: Unexpected end of array

It happens even if I use the UintArray constructor like this new Uint8Array(foo). I have been banging my head since last week solving this error. What am I missing here?

4

There must be something else going on there, maybe with your encoding/decoding, because applyUpdate works perfectly fine in memory:

import * as Y from 'yjs'

const doc1 = new Y.Doc()
const doc2 = new Y.Doc()

const map = doc1.getMap()
map.set('a', 1)

const state = Y.encodeStateAsUpdate(doc1)
Y.applyUpdate(doc2, state) // works fine

In that case I might recommend using one of the supported db providers, like leveldb, mongodb, or redis.

1 Like