Understanding memory requirements for production usage

Hello! I’ve been following YJS development for a few months now and am super impressed with this tech. I run a collaborative platform for writers and RPG gamemasters (legendkeeper.com). I think I have a decent idea of how I’d like to implement YJS for our prosemirror-based editor, backed up by postgres, but I’d like to further my understanding.

My biggest remaining question is about memory requirements on the server side when using y-websocket. In this post, you talk about a large YDoc consuming 40mb of memory. Is the full document always required to be in memory when manipulating YDocs, or is that dependent on the persistence method? Or is it only fully in memory during moments of mutation? Still trying to wrap my head around the mental model of it all. I think YJS fits well into our tech stack, just unsure if it fits within our budget. We have 500 users on at a time and are growing quite a bit, and these users like to open multiple documents simultaneously.

1 Like

Hi @braden,

The main advantage of using Yjs is that you can scale indefinitely using a simple pubsub server to propagate document updates.

The memory usage of representing a conference paper (see this post) is about ~2MB (260k edits). Although there currently seems to be some bug in y-websocket which blows up memory.

I plan to rework y-websocket server not to load the document to memory at all. I explained my plans in the post you mentioned. Eventually, I’d like to be able to handle 100k connections on a single websocket server.

Thanks for the reply! That clarifies things for me.

hey @dmonad how is this rework going? Has this bug been fixed. we seem to also be affected by it?

Thank you

Hi @CoCreator-Frank,

As I explained in the ticket you are mentioning I’m hoping for someone to come in and provide a fix. I really can’t stem this project and my day job on my own. There were a couple of people who offered their support, let’s see if they can fix it first.