Meta-Post: Collection of Yjs Servers and Invitation to Unify Efforts

1

The purpose of this post is, on the one hand, to collect all the individual initiatives of creating a Yjs server, and thus make the research easier for everyone.

On the other hand, it has the objective of organizing us as a community so that we create a definitive Yjs backend. The pieces are already there mostly, we just need to put them together.

Here is the summary of my research:

library Auth Persistence Scale Multi/Sub-documents Protocol-agnostic Persons
y-websocket NO PARTIALLY** NO NO NO @dmonad
y-websocket-auth (issue) (discuss) YES NO NO NO NO @raine
yjs-scalable-ws-backend (discuss) YES* On every update YES NO NO @kapv89
(1) y-multidoc-websocket (discuss)
(2) snippets on discuss, related blog
(3) snippets on discuss 		NO NO NO YES NO @LeeSanity, @braden, @DAlperin, @nokola
yjs-server (discuss) YES YES NO (planned) NO (planned) NO @luishdz1010, @douira
hocuspocus YES YES*** YES NO (planned) NO Âżnot in discuss?

.* it’s in the external_api_persistence branch
** You can get basic leveldb persistence without forking by starting the server with YPERSISTENCE=./dbDir. Also, there is an internal API, but you need to fork the repo. See this, this and this.
**hocuspocus Previously allowed to persist only rocksdb. Now allows any DB.

  1. A sixth important feature that could be added, which Kevin has proposed a couple of times (mainly here, also here or here) is that each document or sub-document keeps a last-modified field and then users when they connect sync those documents that are not up to date. Sounds nice.

  2. Of course, a prerequisite for the server would be to use the new differential updates API so that it can save changes without bringing the entire document to memory. I’m not sure which of the libraries use this API so I didn’t add it to the table. If anyone has observations to update this or any other information please put it in the comments and I will update this post.

  3. A Middleware API for additional server functionality it’s a must too (thanks @raine). If I’m not mistaken, Hocuspocus also has this thing called “Hooks”.

About Hocuspocus: they plan to change the business model, making the project open source, and in return offering optional cloud infrastructure. From what I’ve read, it sounds like they were planning to get that done in the middle of last year, but they’re having some delays getting it done.

In order to make an official open source server a reality, I can think of 2 ways:

  1. We could create a new repository at Yjs · GitHub, if Kevin is okay with it.
  2. Sponsor Hocuspocus or collaborate on its code. Based on what I said above, this would speed up your release to open source, as far as I understand…

open debate :slight_smile:

2 Likes
2

@german-jablo Wow, this is incredible!

I think it would be valuable to have a middleware API for additional server functionality. Forking y-websocket to add a (common) feature is not composable or future proof.

You can get basic leveldb persistence without forking by starting the server with YPERSISTENCE=./dbDir. Limited, but worth mentioning.

1 Like
3

Just a note … my project has facility for auth, it’s in the external_api_persistence branch

4

Thanks @raine and @kapv89! I have already updated the post with your observations :slight_smile:

5

Hi @german-jablo ,

Thanks a lot for including Hocuspocus in the list :pray: I am Philip from Team Tiptap/Hocuspocus. One of our Discord users just made us aware of the channel here.

Feel free to add @janthurau and me as a contact for Hocuspocus.

Two core maintainers left us last year, and we had a lot of challenges in organizing the situation, hence the delays in the stable release (see: News on Tiptap and Hocuspocus – Tiptap Editor). A big sorry from the whole team!

We plan to publish the stable release and make Hocuspocus available for free to everyone under the MIT license soon. Contributions are always more than welcome!

2 Likes
6

We published the stable release of Hocuspocus and made it available for free under the MIT license. Thank you to everyone who sponsored and supported us to make this possible!

4 Likes
7

Thanks to you and the entire Hocuspocus team for the amazing work! Looks great!

I think this was a key piece that was missing from the Yjs ecosystem, I’m very excited.

Note: I can’t mention them in the post above right now, because of forum settings.

3 Likes
8

@philipisik Is hocuspocus a general purpose Yjs server? What I mean by that is how much is hocupocus aware of the frontend it’s communicating to. In the documentation, I see this line - Collaborative text editing (with Tiptap, Slate, Quill, Monaco or ProseMirror). And I’m wondering if it can support other editors or even non-editor frontends, out-of-the-box?

1 Like
9

@chrysalis Yes, Hocuspocus is a general-purpose Yjs server. We plan to communicate this more clearly in the docs later this year.

2 Likes