YArray loses a part of information

1

Hi. I am having some problems with Y-Array on WebRTC. I am doing a peer-to-peer document collaboration project. I am using the Y-Array for sending Operational Transformations. I am creating myself the Operational Transformations, I use Y-Array just to send them between clients. The problem is that, sometimes, when a client joins, some data that was already pushed is slightly altered.

Ex: C1 creates a document and performs some operations. C2 joins the document, gets the operations, and everything is fine. C3 joins the document, gets the operations, and everything is fine. When C4 joins, he gets the operations, but some of the data is lost.

Data looks like this: { p: (3) [5, 2, 7], si: “some inserted text” }. C1 gets this p, but when C4 joins, the p is only [5, 2].

join

//Initializing
const ydoc = new Y.Doc();
let provider = new rtc.WebrtcProvider(webstrateId, ydoc);
let arrayOp = ydoc.getArray('op');
arrayOp.observeDeep(observeFunction);
//Observe function that is used to get the data from other peers and apply the OTs sent
const observeFunction = () => {
	let ops = [];
	let allOps = arrayOp.toArray();

	console.log("Operations Start")
	for (let i = opCounter, size = allOps.length; i < size; i++) {
		ops.push({
			...allOps[i]
		});//allOps[i] = {si: "Hello", p: [5, 2, 10]}                
		console.log(allOps[i].p)
	}
	console.log("Operations End")
	// apply OT - not relevant	
}; 
//Function used to add items inside the YArray
coreEvents.addEventListener('createdOps', (ops) => {
	// Apply OT on JsonML doc, not relevant
	// ...

	// ops = {si: "Hello", p: [5, 2, 10]} 
	arrayOp.push(ops.slice());

}, coreEvents.PRIORITY.IMMEDIATE);

This issue happens only to later clients that join the Y-js room and there are already some OT that took place already. If they are all present at the same time, and there is no document updates prior to their arrival, everything works as expected (all the operations are executed correctly across all clients)

2

I’m fairly confident that Yjs doesn’t just modify existing content. A common mistake is that users modify JSON objects after they insert them into Yjs. e.g.

const obj = [5, 2]
ymap.set('p', obj)
obj.push(7) // you must not do that!

It is not possible to detect whether to detect that you change a JSON object (e.g. {} or []). So the local state will be [5,2,7] but remote users will have [5,2]. Instead of [] you could use a Y.Array to share content and modify it after you inserted it.

A quick fix would be to do

arrayOp.push(JSON.parse(JSON.stringify({
    ...allOps[i]
})))

That said, it might be a good idea to avoid the complexity of implementing an OT algorithm on-top of a sequence CRDT. I’m curious why you don’t simply use Y.Text?

3

I will try that approach and see how it works.
I am not using Y.Text because I am using OTs to modify the entire web page, not just to insert text. There was a project that was doing this, but it was centralized. That version of the project was using these lists for tracking OTs and revert operations. I am using Y-Array to keep this functionality in place.

4

I see. FYI Y.Text completely supports the OT rich-text format. You can use ytext.applyDelta(otOperation) and event.changes.delta for the events. But if you are using other OT types, then you probably have to stick to your current approach.