So I’m starting to try to build a PouchDB provider (I’m using the indexdb one as a starting point) and plug-in for Yjs. My plan is that a Y.doc will manage the whole Pouch document and be included in it as a binary attachment. Then have a top level Y.Map that is also exported to json at save time to build the pouch document, this can then be queried and indexed by the standard pouchdb api. So the user doesn’t ever change the pouch document directly, only the Y.doc, and with that we can get conflict free merges in PouchDB.
When fetching a document from pouch we will also fetch all conflicting versions and will merge the Y.docs. Then we watch the pouchdb change feed and continue to merge in changes as they arrive.
This method obviously only merges whole document versions, great for offline editing but not real-time where vectors are better. It seems a combination of this with the websocket or webrtc provider for real-time collaboration would work well. PouchDB revision keys are deterministic hashes of the document and so if you combine a PouchDB provider with a websocket provider to merge in update vectors the resulting document will have the same revision number, fully sidestepping the PouchDB/Couchdb sync (also important for the normal pouchdb sync). We would probably want to ‘pause’ or slow down the save to PouchDB while using a realtime collaborative provider to stop too many full document syncs in the background.
I’m also considering having an option for a pre-save extractor function, this would allow you to extract additional data (say from the y-prosemirror doc Y.XmlFragment) and add it to the main pouch json doc for indexing and searching.
I intend to opensource/contribute back anything that I can get working on this.
This plan should work well when you have a document open but the area where I have a question is when documents are not open. PouchDB will continue to sync in the background (when the app is open), and we can easily watch the change feed for new version conflicts of unopened documents. When using this with Prosemirror do we need to have the document open in Prosemirror (and have access to a browser DOM) when doing a merge or can we just naively merge the Y.docs? Would this cause a problem with the Prosemirror schema?
Anyway, also just wanted to say Yjs is amazing, having spent the last couple of days reading up on everything I’m seriously impressed with what Kevin had achieved!