Enforcing a data structure

1

I’m making a small note taking application and I really want to use Yjs for the collaborative aspect of it. However I’m stuck on how you can validate the updates sent by clients.

Basically there are projects and each project has cards where users can, for example, type in text. Currently this is all done with a client-server architecture where the server is the source of truth. Clients can update cards and send their updates to the server which will broadcast it to all other clients.

Since Yjs is schema-less I believe this means clients can add arbitrary data to the Y.Doc and the server will just happily add it to it’s own local copy of the Doc and broadcast it. Instead what I want to do is validate the update sent by the client and, only if the update is valid, I want to apply it to the local copy of the doc on the server and broadcast it to everyone else. I want to know how to do this in a nice way.

let Ok(bytes) = general_purpose::STANDARD.decode(update) else { return };
let Ok(update) = Update::decode_v1(&bytes) else { return };
let doc = project_state.doc();
{
    let _ = doc.get_or_insert_map("nodes").observe_deep(|txn, events| {
        for event in events.iter() {
            match event {
                Event::Text(x) => info!("{:?}", x.delta(txn)),
                Event::Array(x) => info!("{:?}", x.delta(txn)),
                Event::Map(x) => info!("{:?}", x.keys(txn)),
                Event::XmlFragment(x) => info!("{:?}", x.delta(txn)),
                Event::XmlText(x) => info!("{:?}", x.delta(txn)),
            }
        }
    });

    let mut txn = doc.transact_mut();
    txn.apply_update(update);
}

This is the current code I have on the server (using Yrs). I was thinking of taking a snapshot of the current document, applying the update, checking if all the changes are valid, and if something is off then reverting back to the old snapshot. This solution seems pretty ugly, like I was expecting maybe you can inspect the update directly to see the things that are being modified and if it’s fishy simply not apply it. Another problem I’m having is how do you check if the update modifies anything on the top-level, like as far as I know I can’t do observe_deep on the whole document. Thanks in advance!

2 Likes
2

Okay looking around it seems like Yjs isn’t really fit for my problem. It looks like Yjs is meant for situations where every client is trusted, where in my situation I have a central server controlling the document, validating every change every client is doing to make sure it’s not malicious.

3

I’m in a similar situation and I’d be interested if you found a colab solution for your notes app?

1 Like
4

I ended up going with a sort of Operational Transform solution for my app. I have designed some operations that are possible on the document and the client sends them to the server. The order in which the server receives the operations is considered to be the “official ordering”. Based on the order received the operations are given a timestamp and then that operation along with the timestamp is distributed to other clients. The timestamps are Lamport Timestamp. (So the timestamp starts at 0 on the server for the first operation and every next operation increments that timestamp.) The clients also do this on their side separately.

Server:
OpX(1) → OpY(2) → OpZ(3)

Client:
OpX(1) → OpZ(2)

In the above scenario the client hasn’t received OpY and therefore assumes OpZ comes right after OpX. When the client receives OpY the client can conclude that OpY must have come after OpX as to not contradict whatever the official ordering is. (OpY cannot be before OpX because then OpY would need to have timestamp 0 and not 2, OpY cannot be after OpZ because then OpY would have timestamp 3 and not 2, therefore OpY must be after OpX)

Leaving the client with:
OpX(1) → OpY(2) → OpZ(3)

Which is the same as the server, therefore the operations have been synchronized and so the same document can be rebuilt on both ends.

(Now I haven’t actually implemented this yet, but I think this should work, and perhaps my idea might be flawed here but I’m not sure)

1 Like