Yjs as a bidirectional client + service layer in a microservice framework(?)

I’ve been thinking about this for a week or two, and it finally came out in words…

I’m considering using Yjs for browser extension… the browser extension background.js has listeners set for messages from content tabs and from the chrome APIs. It writes them in an event store append-only array which is broken up semi-regularly into subdocuments, so the entire history only has to be loaded into local memory as necessary or to the last state snapshot (using event sourcing for this particular project)

The NodeJS server, hocuspocus, in this case, take these events and passes them to a service broker which takes care of the transport to the message broker and to the persistence service, analytics, whatever. The hocuspocus/gateway service is a producer in this case. It is also a consumer of some aggregation services or whatever is needed and this is automatically passed back down from the service broker’s reception, into a subdocument corresponding to the particular service to which we are subscribed. Handy eh?

I’ve essentially replaced an APIful gateway service with an API-less Yjs streaming service that sucks up my events and hands me back down aggregates or additional event log entries (as updates) as needed. I don’t have to do anything but push up events and react to the changes in state. I think this is a good idea. Haven’t tested it yet.

(Obv, reception of new events would trigger an update to the state which is only a projection of the event log; notably, the state is not sent to the backend, only projection snapshots of state corresponding to a particular master event IDs are received from the snapshot service as a sanity check. When we make or receive a new snapshot, we end the event log at that ID and start a new subdocument such that each snapshot is corresponds with each event log subdocument)

I wonder if Yjs would be a good fit for an entire microservices’ backend? I mean, why stop at the frontend? :thinking: I cannot say, as I am most certainly not planning on implementing it. :stuck_out_tongue_winking_eye: Perhaps it’d be some kind of distributed stateful orchestration microservices type deal, like https://cloudstate.io/(?)

So… sanity check-- is there a problem with mixing strong consistency of event sourcing and eventual consistency of CRDTs like this?

No, provided the events log is append-only.(?)
“In Event Sourcing, state in is the event log while state out is any newly persisted events as a result of handling a command.” (link)

1 Like

@dmonad I was wondering if you had some specific advice regarding the implementation of append-only event logs used for event sourcing and the suitability of Yjs when used in this way? Would be curious to hear your thoughts.