Advanced apologies if this question was addressed before.
How is Yjs integrated with editors (say SlateJS) to enable collaborative editing?
I assume the desired approach is to make Yjs the “system of record” and editors act as UI clients that accept input from local users. Specifically, if there are two nodes, the editor of the first node will be connected to one instance of Yjs doc, and the editor of the second node to the second instance of Yjs doc. Local operations will be accepted by each editor and conveyed to its connected Yjs doc. Remote operations received by the Yjs doc are conveyed back to the editor.
Assuming the above setup is right, how is an editor and its corresponding Yjs doc kept in sync at all times? It is conceivable for a (human) user to be inserting characters into the editor at a certain position in the sequence while its Yjs doc is receiving concurrent operations from a remote node that affect the position where the new characters are being inserted. In other words, the editor also becomes a source of concurrent operations.
I can imagine a few solutions:
Pause merges of remote operations at Yjs doc when local operations are flowing in. This will prioritize local operations, but other than occasional jitteriness, positions implied in local operations are always meaningful because all remote operations seen by Yjs doc are also seen by the editor (and therefore the human user). I believe this approach is the most practical, and one could pause merges by “going offline” or find a way to somehow tell Yjs to not accept remote operations temporarily.
Replace the editor’s internal data model with Yjs doc’s data model. This way, both user inputs and remote operations are changing the Yjs doc directly. And Yjs is already capable of handling concurrent operations. However, it is usually impractical to replace editor’s data model unless we have access to and willing to change the editor’s source code.
Allow users to enter characters via the editor, but intercept those events and send them directly to Yjs doc instead of to editor’s internal data model. The output from the Yjs doc is what is fed to the editor model for purposes of displaying the text back to the user. This approach can be implemented in two ways and both seem farfetched. One implementation is to update the whole text in the editor by reading back the full string from Yjs doc for each character change. The other implementation is to calculate the positions where text got affected from the recent local/remote operations at Yjs doc, and transform those Yjs operations into editor-specific operations and apply them on the editor. The second implementation sounds a lot like a mini-OT solution, which is not preferable.
How is the solution implemented in practice? I appreciate your thoughts.
There are a couple of approaches to making an application collaborative (i.e… synchronize an editor, or a drawing app with Yjs).
If you are building a custom application (e.g. a drawing application), then it makes sense to make Yjs the only source of truth. Whenever you draw a line, you will manipulate the Yjs document. Yjs will fire an event and you reflect the changes on the canvas. This is the easiest approach.
There are a lot of existing editors with a custom editor model that Yjs can make collaborative. In this case, we find an equivalent representation of the editor-state in Yjs. A Quill editor, for example, uses the Quill Delta format (basically text with formatting attributes over ranges of text) as a representation of editor content. The Quill Delta is synchronized with a Y.Text data type.
An editor binding is responsible for keeping an editor state in sync with a Yjs shared type. Existing implementations are: y-quill, slate-yjs, y-prosemirror. Whenever the editor changes, we reflect the same changes to a Yjs type (we use the computed deltas of the editor). Whenever the Yjs document changes, we reflect the changes to the editor instance (transforming Yjs deltas to manipulations on the editor). I refer to this approach often as “double binding”.
Pause merges of remote operations at Yjs doc when local operations are flowing in. This will prioritize local operations, but other than occasional jitteriness, positions implied in local operations are always meaningful because all remote operations seen by Yjs doc are also seen by the editor (and therefore the human user). I believe this approach is the most practical, and one could pause merges by “going offline” or find a way to somehow tell Yjs to not accept remote operations temporarily.
This is never an issue as JavaScript is synchronous. Remote operations are applied synchronously, and fire an event synchronously, which will immediately update the editor content.
I believe it depends on how the editor.apply(ops) function is implemented by the editor folks. If the editor does not synchronously apply the operations received from its Yjs type, then the editor and the Yjs type would diverge. Right?
In the case of Slatejs, it appears editor.apply function is synchronous, so that is not a problem with Slate.
That’s right. However, operations can always be applied synchronously. There were some editors that cached operations for a time before committing them to the internal data model. However, they all implemented a flushOperations method that force-executes the operations. Generally, I’d say, that these kinds of “optimizations” are harmful and lead to bugs, so modern editors avoid them.