Garbage Collection and Version Snapshotting

[braden]

In my production app with millions of Y.Docs, we have a certain type of domain model entity that uses Yjs with garbage collection turned OFF because we utilize version snapshotting.

This tradeoff has not been a great one. While the snapshots are nice, the GC is far more useful. We’re finding docs with GC disabled are pretty awful for performance, disk space, and network throughput. I would advise anyone thinking about doing the same to strongly reconsider.

If I just turn GC back on for all these docs, anyone using the snapshots will get screwed over, but the ramifications of turning GC off have been bad enough that I’m considering it. Does anyone have any ideas for using these features together gracefully? There was talk of being able to use both, but I can’t find the example, and from what I remember it seemed like an “off the books” kind of thing.

[braden]

I found a compromise:

Our backend now garbage-collects everything, no matter what. The client garbage-collects anything that isn’t exposed to the versioning feature, and now we store the snapshots in the browser, rather than in the Y.Doc. We no longer offer fully networked versioning, only local version snapshots that are lost when the cache is cleared. No one seems to mind, as most usage of the version snapshot feature was recovering accidentally deleted data after the session had ended.

EDIT:

Before and after enabling GC on the server. The CPU spikes are from database writes of oversized, non-GC’d documents. Enabled GC on the 11th.