Delete set pruning based on a target state vector

#1

First, thank you for the fantastic library and supporting ecosystem. It’s great!

I noticed that createDeleteSetFromStructStore does not prune the deletion set based on a target state vector the way writeClientsStructs does. Is this by design or would delete set pruning based on a target state vector be something you would be open to a PR for? I think that this should be sound but wasn’t sure if there is some edge case you had in mind for which similar pruning of delete sets wouldn’t work.

I ask because I am building a collaborative editing app using prosemirror and yjs with a central server. To ensure that the full state of the doc is sent to the server even in the face of database/server failures, I am maintaining a client-side view of the latest server state vector and sending the server updates diffed against that server state vector. Because delete sets aren’t trimmed, I’m seeing the client send updates that are constantly increasing in size with each deletion even though the server has already acknowledged those clocks.

I’m happy to put up a PR if this is something you would be open to.

#2

My limited understanding is that the compressed delete set is so small on the wire that additional optimizations were judged to be unnecessary.

Based on How Yjs works from the inside out - YouTube

#3

Thanks! Great video. Based on this, it sounds like my understanding was incorrect. Based on the video, there is no concept of incremental/delta deletions and all deletions need to be sent up front.

I’m up to 4kb of deletion data after a few days of testing on a doc so will keep an eye on how this grows.