I’m currently storing base64 representation in my db. The base64 doc size is significantly larger (e.g. 10x) than the simple delta representation. I can reduce the base64 size by converting to deltas and creating a new doc, but even then the base64 size is still larger (e.g. 1.5x) than the delta.
If I’m already going thru the process of converting my base64 to deltas (to reduce the size), I’m thinking I might as well store the stringify version to the db. I understand the base64 is preferred for transport, so I would still need to convert the deltas back to base64 when retrieving the document.
The benefit of storing deltas are: 1) smaller size 2) readability
My process would change from:
- running cleanDoc below before storing a base64 doc
to:
-
running convertBase64toDelta when I received a base64 doc
-
running convertDeltaToBase64 when I retrieve the deltas from the db.
From a server cpu processing time, I think it’s somewhat of a wash. Right?
Am I missing something? Is there any reason to store the base64 version?
function convertBase64toDelta(ydocBase64: string): string {
const yDoc = new Y.Doc();
Y.applyUpdate(yDoc, toUint8Array(ydocBase64));
return yDoc.getText().toDelta();
}
function convertDeltaToBase64(delta: any): string {
const yDoc = new Y.Doc();
yDoc.getText().applyDelta(delta);
const unit8Array = Y.encodeStateAsUpdate(yDoc);
return fromUint8Array(unit8Array);
}
function cleanDoc(ydocBase64: string): string {
const yDoc = new Y.Doc();
Y.applyUpdate(yDoc, toUint8Array(ydocBase64));
const docDeltas = yDoc.getText().toDelta();
const yDoc2 = new Y.Doc(); //redundant but necessary
yDoc2.getText().applyDelta(docDeltas);
const unit8Array = Y.encodeStateAsUpdate(yDoc2);
return fromUint8Array(unit8Array);
}