[Prosemirror] Data lost after creating a Heading

Let’s say we have clients Alice and Bob.

  1. Alice is online, Bob is offline
  2. Alice enters # heading
  3. Bob enters 123
  4. Bob goes online

Now Alice receives Bob’s changes, but they’re discarded, only Alice’s heading is visible.

Here’s a video showing the issue: YJS/ProseMirror sync issue - YouTube

Could it be related to this: Offline, Peer-to-Peer, Collaborative Editing using Yjs - #34 by dmonad - Show - discuss.ProseMirror ?

This is, unfortunately, the expected behavior. The same would probably happen if you’d use prosemirror Collab.

Basically, the left client deletes the last paragraph and transforms it to a headline. The right client inserts into the paragraph that the other client deletes.

Prosemirror has a nested document structure. If you delete the parent you must also delete the children…


For offline scenarios you basically always want to check the changes that you merge before you commit them.

Kevin, thanks a lot. Makes sense now.

The same would probably happen if you’d use prosemirror Collab.

Just to be clear, prosemirror collab doesn’t have this issue. The video below show the same operations with PM collab.

Kevin, it sounds like merging changes to a tree structure is very difficult/impossible.

In this case we were thinking of rewriting our list implementation to avoid using a tree. there will be no list nodes, and ever list-item node will be at the same hierarchy (but have their own ident property).

Are we thinking about the problem in the right way?

I just wanna clear up that this question talks about 2 different issues:

1. Playing with list indentation - one client dedented, and then again indented a list item. This basically destroyed the list and created a new one. Meanwhile, another client made a bunch of changes to that list. However, the list is gone now.

This would be solved by using flat structure for lists. The indentation would just modify property of that list item, it wouldn’t destroy and recreate the list.

2. Converting empty paragraph to heading/list - one client creates a list item in an empty paragraph. Doing this destroys the paragraphs and created a list in its place. Other client writes into that paragraph, but the paragraph is destroyed.

Using flat list is of no use here. This should be resolved in another way - for example maybe the document structure shouldn’t even allow for such thing as empty paragraph. Or typing into an empty paragraph should destroy that paragraph and create a new one.

one client dedented, and then again indented a list item. This basically destroyed the list and created a new one.

Changing the way how the document is represented in ProseMirror can help with this. If the list is represented as a list of <li depth="x"> allows users to concurrently change the depth field while others are working on it.

This would be solved by using flat structure for lists. The indentation would just modify property of that list item, it wouldn’t destroy and recreate the list.

Exactly!

one client creates a list item in an empty paragraph. Doing this destroys the paragraphs and created a list in its place. Other client writes into that paragraph, but the paragraph is destroyed.

The 1-on-1 mapping to prosemirror nodes is not ideal. Currently, we do have to delete the empty paragraph before we insert something of a different type. One way around it is to introduce a single <typography> tag that also works as a paragraph and as a list item by changing a single property.

I realize that this is not ideal, but we have to work within the constraints.

for example maybe the document structure shouldn’t even allow for such thing as empty paragraph. Or typing into an empty paragraph should destroy that paragraph and create a new one.

That would have other bad consequences. Other people might expect that everyone always sees the same content (even empty paragraphs). Deleting a paragraph and creating a new one seems like a good idea. However, you might still destroy unsynced changes to that paragraph.


My arguments for a lot of these changes is that:

  • Automatic merges can never be perfect (maybe if you’d introduce an AI), they can only get slightly better.
  • These kinds of merge conflicts don’t matter in realtime-collaboration mode (messages are exchanged within a few hundred milliseconds).
  • For merging changes produced while offline, you definitely want to compare the changes before merging them. I feel this is the real missing feature. Unfortunately, this is not yet natively possible in Yjs yet, but we will get there.

@dmonad Would you think it’s practicable to duplicate content when such conflict happen?

Use the example in the topic description, that means keeping both the heading from Alice and the paragraph from Bob when merging. This can lead to some duplication of text, but will minimize (if not eliminate) losses.

If it’s practicable, how do I implement such strategy?

Thanks!

You have to work within the boundaries of the Yjs CRDT. I suggested reworking the document model so that more efficient merges can happen. I don’t know how a duplication approach would be implemented in any CRDT.

Thanks. That would be very helpful. We do plan to change the document model. Just want to try some other approach in the same time.