Order in which Events yielded by observeDeep should be applied

I’m working on some code that uses a shared YArray to synchronize data between clients. I’ve built a test framework that allows me to:

  1. make application-level changes in one client
  2. capture the YUpdates generated by the Yjs layer for that client
  3. apply those YUpdates to the Yjs layer for a second client
  4. use .observeDeep() on the shared YArray to invoke code that propagates the effect of those changes to the application level in the second client
  5. validate that state meets expectations at each step in the process

This is mostly working great, etc.

But I’ve run into a specific situation in which moving from step (3) to step (4) involves .observeDeep() being called with both a TextEvent and an ArrayEvent – specifically, one in which the ArrayEvent is adding nodes to the Yjs hierarchy in a way that affects the path that should be used for the TextEvent.

In the current state of things, it appears that the TextEvent always precedes the ArrayEvent. However, it further appears that the path encoded in the TextEvent reflects the state of the world that would result after the ArrayEvent had been applied – e.g., meaning that processing the Events in the given order (TextEvent, then ArrayEvent) yields incorrect behavior (either applying the affect of the TextEvent to the wrong Yjs element, or possibly throwing an exception because no Yjs element can be found at the specified path) – meaning that although validation succeeded at steps (1) through (3), it fails at step (4).

I can also reproduce this with a test case that yields a MapEvent instead of a TextEvent.

I suspect what this means is that when I receive a batch of Events via .observeDeep(), I need to have some policy for the order in which they should be applied to the application-level representation (i.e., other than the ~obvious choice of apply them in sequential order, as that is yielding incorrect behavior). One example of such might be “always apply ArrayEvents first, then other Event types” – but while that would address the problems I’m seeing in these test case (and not cause any of my other existing test cases to fail), it also feels like a potentially ad hoc solution based on a limited number of examples.

Ergo my question: When .observeDeep() is invoked with more than one event, what should I assume about how those events relate to one another and how they should be applied to the application-level representation? Is there a static policy that I can use (e.g., the aforementioned “ArrayEvents before other Events”), are there annotations that I can use to construct an appropriate order, or something else?

Hopefully I’ve provided enough context, but happy to answer questions if I haven’t.

Thanks!

I may have made some progress on this.

Empirically, it appears that when the handler registered with .observeDeep() is called with multiple Events, Yjs has already adjusted the path(s) for those that operate on elements deeper in the document hierarchy as if any Events that operate on elements more proximate to the root of the document hierarchy had already been applied – even if that does not match the order in which the Events are sequenced in the array passed to the handler.

I’ve verified (empirically) that this “path adjustment” is happening for simple cases involving ArrayEvents, TextEvents, and MapEvents (when combined with an ArrayEvent that might affect the path(s) associated with those events).

Ergo, it feels like a policy of sorting events by path length before applying them to the application-level representation would be appropriate – and doing so does fix the failing test cases of this sort that I’ve identified without breaking any other test cases.

@dmonad – any thoughts on this? Thanks!

Hi @kjohnson,

thanks for figuring this out! I haven’t thought about this case. I like the solution to sort events by the path lengths.

I implemented your suggestion in https://github.com/yjs/yjs/commit/4fb9cc2a308d21fde3a1b8567aaec36a85820ef1

I’m currently working on another issue, but by tomorrow you should be able to update to Yjs@13.4.2 where this issue is fixed.

Hello @dmonad

Thanks for taking a look and incorporating the fix – much appreciated!