My adventure in building a complicated application on top of yjs subdocuments

Hi all! Over the past two months or so I have been prototyping a new project in which a core feature is real time collaborative editing. Yjs has been really awesome to work with.

I recently wrote up some of the challenges and solutions I decided on when I came up on the edges of what the ecosystem supports, namely sub documents and robust server side persistence (in your application database).

I’d love some feedback from folks who might have been working on other approaches. I had a super interesting conversation with someone who found and referenced my y-websocket fork about how they approached some of these issues. I think it’s definitely worth a read, as they went a totally separate direction: Multiple text types per doc + multiple editors in the UI · Discussion #310 · BitPhinix/slate-yjs · GitHub

My y-websocket fork with the changes to support multiplexing subdocuments over one connection can be found here: GitHub - DAlperin/y-websocket: Websocket Connector for Yjs I hope to find some time to release an example compatible server implementation as well.

My y-prosemirror changes can be found in this open PR here Introduce sub document support by DAlperin · Pull Request #88 · yjs/y-prosemirror · GitHub those changes are pretty uninteresting, just fixing up a lesser used feature which is key to subdocuments as well as adding some compatibility options to make slowly migrating to subdocuments easier (in theory)

Let me know what you think and if you have any questions!

4 Likes

Really awesome work @DAlperin!

I appreciate your contributions and especially your blog post. Thank you for sharing your experience!

I’m sorry that I haven’t had time to go through your PRs. The y-prosemirror changes seem reasonable (I will write about my intentions for the current approach in the PR). The y-websocket changes make a lot of sense. But I probably have to reject it because it breaks the current authentication approach. Most users of Yjs implement authentication by authenticating each WebSocket connection using URL parameters. The subdocument protocol addition will likely break that and allow users who have access to one document to have access to all other documents. However, your implementation is fine and I’m happy that you made that work in a separate provider.

Personally, I also think that subdocuments are one of the coolest new feature. Thanks @braden for funding that! So far we are missing a scalable backend that can sync subdocuments. You’re right that I’m working on a really cool new protocol that allows syncing and authenticating subdocuments. But that’s all secret :shushing_face:

Thanks for making yjs! It’s made what would otherwise be one of the most complicated part of an already complicated system relatively effortless. Subdocuments are definitely a ridiculously cool feature (thank you @braden). I figured you’d have something cool in the works for a more right way to sync subdocuments. My implementation is pretty crude but it works, I also didn’t have to worry about breaking compatibility since my closed source server implementation is pretty clean room and does some weird stuff.

I wish I could help sponsor development but alas I’m a student. If there is ever any way for me to help with the official subdocument protocol let me know.

(Ps. I’m sorry about my open y-prosemirror PR, it’s a little disorganized)

2 Likes