I believe @dmonad is working on a significant update to y-websocket. If his update doesn’t include multiplexing support, I’d be happy to adapt my solution and submit a PR once the update goes out. As it stands now, my fork of y-websocket has diverged greatly from the current version of y-websocket and would be made irrelevant once the new y-websocket goes out anyways. That being said, in the meantime I can outline a loose solution:
Add new message types such that the full y-websocket protocol is:
message formats:
[messageSync][messageType][data]
[messageSyncSub][subdocId][messageType][data]
Extend websocket provider on the client and WSSharedDoc on the server to have a data structure that tracks loaded subdocs.
class WSSharedDoc extends Y.Doc {
...
subdocs = {}
...
loadSubdoc(subdocId) {
let subdoc = getYDoc(subdocId)
this.subdocs[subdocId] = subdoc
return subdoc
// or return an already created subdoc
}
}
const messageHandler = (conn, doc, message) => {
const encoder = encoding.createEncoder()
const decoder = decoding.createDecoder(message)
const messageType = decoding.readVarUint(decoder)
switch(messageType) {
case messageSync:
// same as default y-websocket implementation
case messageSubSync:
let subdocId = decoding.readVarString(decoder)
let subdoc = doc.loadSubdoc(subdocId)
// now that we have the subdoc, the sync process becomes identical to the standard case.
// be sure to broadcast any changes uses your subsync message type
syncProtocol.readSyncMessage(decoder, encoder, subdoc, null)
send(doc, conn, encoding.toUIntArray(encoder))
}
}
When we detect a subdoc sync message, we just extract the subdocID, and then from there, the sync process becomes identical to a non-subdoc. There’s probably a clever way of reducing code re-use here, but I haven’t refactored much. Regardless, it works very well and can sync an enormous amount of subdocs very quickly.
The client does the exact same thing. Add a subdocs collection to WebsocketProvider, when we detect a subdoc message, use the subdoc ID to load the new Y.doc into memory, and then we use the contents of the message to apply updates to the doc, just like a non-subdoc Y.Doc. A lot of these details change when you introduce YJS’ newest features, mergeUpdate and diffUpdate, but I’m still getting acquainted with the api.
