I’m building a note-taking app with Lexical, and I’m building a feature that will allow users to query in document A for content that lives in document B, and see and edit it within document A. In practice it might be multiple documents whose content is included - so you can imagine a document that has a node where the contents are a formula, and that formula returns everything with a specific tag. This might result in pulling content from many documents.
Ideally, users would be able to add these query nodes anywhere in a document
I’m planning a separate in-memory search index / query language that will point to contents managed by yjs
I’ll either check for circular references or just exclude query results from the search index
Does anyone have tips about how to structure this within yjs? eg in the case where Doc B content is included in Doc A, should Doc B and Doc A both be subdocuments of another master doc?
The major problem I see with this is that you need to connect to a large number docs while editing a single doc. Of course, you could connect lazily so that the Doc A content within Doc B would only be real-time when user activates it / scrolls down to it. But it’s going to be awful lot of connections pretty fast, I imagine. Well you can multiplex them but still the server needs to broadcast them to everyone all the time.
Other than that, it seems doable although time-consuming. Probably easier would be to make the doc A content static, allowing for easy GET requests and only turning them into Yjs docs if they are edited. And even that probably only shown when user actually activates the cell.
YJS is good for a fixed number of Docs, but not so good for n Docs in my experience.
The subdocument feature provides only the thinnest level of support for multiple related Docs. It is up to you load, unload, and sync subdocs as needed.
I say this having built a note-taking app with one Doc per note, custom lazy loading, prioritized replication, multiplexing over websockets—and run into massive memory issues at around 100k notes with very little history. And that’s after giving up atomicity for edits across multiple Docs. I would recommend avoiding YJS for n Doc scenarios.
Thanks for the thoughts. This is a single-user scenario, so I don’t need to worry about simultaneous editing / sync, so maybe there’s a simpler solution.
(Users can edit their docs on multiple devices, so I will have to worry about sync, but can probably handle that separately - probably with yjs.)
Multiple devices is a similar problem to multiple users. However it can potentially be solved with a simple conflict resolution scheme such as a latest-timestamp or last-write-wins.
I wish CRDT technology was more performant for large graphs, but I don’t think we’re there yet.