I am trying to build a custom provider for lexical & i’m using custom backend server (socketio). I am stuck could not find any resource on how to create a custom provider & i’m not sure what interface is lexical YJS expecting
Hi, welcome.
This documentation seems to show the interaction between Lexical, the Y.Doc, and YJS provider quite well. Does this help?
Just as a point of clarification, a YJS provider is typically used to sync a Y.Doc to a network or persistence layer, regardless of how that Doc is connected to the UI.
Lexical ⟷ Y.Doc ⟷ YJS Provider
Are you trying to build a YJS Provider for syncing, as above, or are you trying to encapsulate the Lexical ⟷ Y.Doc binding? I’m not clear about your use of the combined term “editor/socketio” since they are decoupled.
I will be using Socketio as transport layer which means i won’t be encoding/decoding type of messages the way y-webosocket does I think just send the updates/stateVectors over socketio maybe base64 encode them.
I am trying to build custom provider which will use socket-io-client. The way i’m trying to approach the problem by sending the state vectors (server will act as a router and save the updates to mongodb provider).
Do i just register in my socketio-client event listener & register callback to apply whatever changes incoming from the server to the local doc inside the provider would that work? cause i tried to use this implementation of socketio with lexical didn’t work and I tried this implementation which works with monaco editor but didn’t with lexical
Lexical part :
I am confused about what kind of interface does lexical expect what attributes/functions do i need to expose for lexical to work with my custom provider or this part doesn’t matter as long as i register my socket events & process the updates correctly?
Reasons for this approach :
- To reduce memory foot print (large document) probably premature optimization to be honest i will be using nestjs & socketio all these technologies are inherently not performant
- I can plug redis adapter for socketio & automatically be able to scale horizontally
If this approach is sub optimal i’m open to suggestions and feedback.
You’ll have to look at the Lexical documentation, such as the page I posted above, to see how Lexical interfaces with the Y.Doc. Lexical does not need to interface with your network provider. (Separation of concerns!) Only the Y.Doc interfaces with the network provider. When a message from the provider arrives, it will update the Doc, which will trigger an update event, which the Lexical editor will be subscribed to.
I’ve never built a YJS network provider myself, but I’m guessing it’s a whole lot more complicated than that. Take a look at the source code for y-websocket. Presumably you would need to implement something similar in your provider. The client-side code is about 500 LOC, and the server-side code is about 300 LOC. That’s pretty small as far as software goes, but it’s not trivial by any means. It needs to handle connection, reconnection, disconnection, sync, and possibly awareness.
You could fork y-websocket and try to sub out the custom socket code for Socketio. That might be the shortest path.
Not to mention Socketio is heavier than the custom socket library used by y-websocket, and base64 is slower with a larger payload than the existing encoding protocol… it seems like a lot of work for a worse version of y-websocket or the just-released y-redis.
Anyway, I don’t mean to be discouraging. Hopefully there is something you know that I don’t know that makes all of this worth doing. Good luck! 
2 Likes
Update :
I have successfully built a POC provider version using SocketIO. I will publish a package and document how to use it once I work out a few bugs in the public API. I will support Redis Streams for infinite scaling and authentication. It works with lexical. I still have a bug when I undo/redo; changes don’t propagate. I think I know how to fix it
2 Likes