Hey @junioriosity - interesting, I hadn’t seen that repo yet. I haven’t read the code in depth so I don’t know if it would provide horizontal scalability in the sense of adding capacity (e.g. increasing maximum concurrency). I also don’t yet understand the purpose of redis queues in that library; given how the queue key is constructed in getDocUpdatesKey (src/redis.ts) it looks like the queue key depends only on the doc ID, and does not vary per server, so I don’t see how this would provide a sort of backpressure etc… yeah, I don’t really get what this library is trying to do with the queues.
For the library’s usage of redis pubsub, it appears to provide similar functionality to Extensions – Tiptap since each message is replicated to every attached yjs websocket server, and there is no sharding/routing, then all the servers should see same/similar CPU and RAM usage and the library does not appear to provide any sort of performance scalability (i.e. “add more servers to handle more concurrent docs/users”). It does provide fault-tolerance/high-availability (HA) benefits: if one WS server goes down, other(s) can continue to accept updates. Now redis-server is the single point of failure unless you use redis-cluster or something similar… but still, this is all about HA rather than perf/max concurrency.
Generally I agree with @dmonad’s advice:
It is probably easier to start with spawing a y-websocket server on a single VM instance.
until you’ve run some load testing to understand where there is a bottleneck for your particular app’s usage patterns. You can vertically scale up a single server quite a lot.
If I were designing a system to scale out to accept a large number of concurrent users, I’d probably do some kind of sharding/routing system. I don’t know of a Yjs websocket library offhand that does this, although there may be one. There are also the serverless approaches, e.g. Y-serverless: AWS Lambda + DynamoDB to use as a provider - I haven’t read those very carefully to understand what tradeoffs they bring.
Hope this helps - good luck with your application!