Troubleshooting Y.applyUpdate (not doing anything)

1

Hello,

I am working on a project where I need to create a custom Y.js provider. My issue is somewhat related to this other post, but distinct enough that I thought a new thread might benefit others who encounter a similar challenge in the future.

In my endeavor to build my own provider, I’ve been utilizing the function Y.applyUpdate(doc, update, this) to apply updates received from my websocket connection. However, despite calling this function, no updates are being reflected and there are no logs or error messages to shed light on why this might be the case.

Here’s a snippet of the update I am attempting to apply:

Uint8Array(20) [1, 1, 207, 253, 224, 198, 3, 217, 41, 132, 207, 253, 224, 198, 3, 216, 41, 1, 49, 0]

After decoding the update to ascertain its correctness, it appears to be fine (I typed 1 from my browser #1):

{
    "structs": [
        {
            "id": {
                "client": 953695951,
                "clock": 5337
            },
            "length": 1,
            "origin": {
                "client": 953695951,
                "clock": 5336
            },
            "left": null,
            "right": null,
            "rightOrigin": null,
            "parent": null,
            "parentSub": null,
            "redone": null,
            "content": {
                "str": "1"
            },
            "info": 2
        }
    ],
    "ds": {
        "clients": {}
    }
}

I am currently at an impasse and am unsure of how to debug this issue further. Is there a way to determine why an update might be “skipped” or not applied? Are there any common pitfalls or additional logging/debugging methods I should be aware of when working with Y.applyUpdate?

I am using Yjs version 13.6.8 and testing this in Chrome 117.

Any insights or pointers would be greatly appreciated.
Thank you in advance!

2

It all depends on what Doc you are applying it to.

Say you create a new Doc and make changes so that it has updates [0,1,2,3,4]. If you take those updates and apply them to a different new Doc you’ll get the same content. You can apply the updates in any order. If you attempt to apply the same update twice, it will have no effect.

However, if you took update 3 and tried to apply it to a new Doc, nothing would happen. If you applied updates 0, 1, 2, and 4 to a new Doc, you would see the result of [0,1,2] (I think… I could be wrong here).

So I would guess that your problem involves either a missing update, or misapplying an update that does not “fit” into the updates of the destination Doc.

1 Like
3

Thank you, I appreciate the helpful answer. I have a situation where I load a document from a database when the page opens, and then I want to start using Yjs for real-time changes right after it loaded the initial content.

I’m making my editor with TipTap and at first, I use setContent to put in the initial content.

I tried using @hocuspocus/provider and it works well, but I don’t know how to do the same thing with my own provider. I thought @tiptap/extension-collaboration package would help, but since I use Editor.setContent myself to get the content from the database, now I’m not sure.

I’ll look into it more and will update this post if I figure it out or if I need more help.

4

I don’t know if you can do that correctly, it will always be problem of your db is not in sync with realtime updates.

I think you always want to get the data from the collaboration extension, if you are using hocuspocus then use the onLoadDocument to send the initial version of the document, and the client will pick it up from there.

5

Thank you for your reply.
I actually want the data to come from the Collab extension, through my own provider, and I’m probably not correctly loading the initial version of the document. I’ll check out how hocuspocus is doing it and I’ll try to replicate the behavior.

6

Hi everyone,

Quick update from my end, I’ve been able to make it work (thanks to your pointers) :tada:

There were two issues:

  • the Yjs document was always out of sync because there were some default values added to the doc
  • I was not applying the updates in the correct order (it was kind of random)

Now, it works well 99% of the time. There are still some weird cases where the new updates in a doc won’t be replicated on reload or on other users screens. I’ll need to reproduce this case and investigate why, but I was wondering if there is a way to detect if an update is applied to a doc or skipped? That would help me a lot to add logs while I’m adding the feature as beta.

Thank you again!

7

Actually its recommended you use the a offline first provider such as y-indexdb in conjunction with your own provider as Yjs supports multiple providers, and always sync the data from local db, and then write the changes to local db when there are remote changes.

Works seamlessly with Tiptap and prosemirror.

8

That’s a very good point, that would make the application much more stable. Thanks for the suggestion.
Are there any examples of two providers working together (or any documentation)?

9

yjs/README.md at main · yjs/yjs (github.com)

10

Thanks, but I’m not sure to understand how it works.
Is the idea to load the document from indexdb first (if the document was already loaded in the past) and then load the document from the remote DB from the second provider?
I’m assuming both providers directly listen to the document updates directly and don’t directly connect to each other, is it correct?
Sorry for all my questions, I’m new to YJS and I want to make sure I make it right. Thanks for your help :pray:

11

That is correct. yjs is magic to me sometimes.

12

Great!

Default values are a little tricky in YJS, since there can be a race condition with other clients/providers trying to set defaults at the same time. Usually this results in duplication though. Not sure why your docs are out of sync.

I don’t think there is. It may be possible to detect discontinuous clocks or something, but that’s not built in. There is probably a bug in your code that is causing an invalid update to be applied to a Doc, or an update being missed due to network failure.

The order doesn’t matter. Once you instantiate the provider, it will sync to the Doc, which will sync to all other providers.

13

Not sure either. What was happening: my app was setting default document content before the document updates come from the server. As the provider listeners were ready, it was actually creating updates to the empty document before anything comes the server, and then when the server updates were applied, nothing happened.

It seems to work now, but just in case something bad happens, I wish I could inform the user. Because if the updates are created, sent to the server, but out of sync, that means the work of the current user would be lost without realizing it (until they refresh the page).

That’s what I figured out, it seems to work nicely. I had to add a little bit of logic to merge document states if a user looses internet connection. In this case, the indexedDB updates would still be saved, while the websocket ones would be lost (if the internet connection is gone and the user leaves the page). But it also seems to be working great now from what I tested.

1 Like