I have run into trouble with y-leveldb, which I am using to persist docs with y-websocket. I’ve been using it for some time and I now have a levelDB of almost 9GB. My server node has suddenly started to balloon, using up to 7GB of memory, slowing y-websocket to a crawl. If I remove the persistence parameter YPERSISTENCE=./dbDir it all works well, so it must the the levelDB persistence that is causing the problem. Has anyone encountered this? And what can I do about it? I could in principle use another persistence mechanism, but how do I then transfer the contents of the levelDB to that?
After a lot of head-scratching, I replaced the dbDir directory with a backup, and now it all works again. Presumably, the database had become corrupted in some way, although there were no visible error messages.
the y-websocket default provide leveldb only support one node by default and hard to horizontal scaling. you can consider migration to mongodb or other easy scale persistance instance.
I agree that that would be good to do. Has anyone had experience of ‘migration’ and how best to do it (and even some code)? I have to preserve the Y-Docs that are in the levelDB database, moving them to some other database. It is not obvious to me how to do this (note that I don’t know the room-names in which the yDocs are stored, so as a first step I would have to iterate through all the database keys).
you can write a small piece of code to migration the data, the official provider the mongodb persistance, also provider y-redis persistance, but the y-redis could not use as commercial, you should buy the license. more info: Y-redis: An alternative backend to y-websocket for me I would consider to migration to mongodb for more easy to scale.