YJS and socket io implementation

Show
1

Hello everyone, first of all I would like to thank @dmonad and this entire community for the great work done with yjs and its entire ecosystem. Second, I would like to clarify that my English is not perfect so I apologize if I make any mistakes. I’d also like to clarify that I’ve never published npm packages before, much less hybrid packages (so I’m pretty new to this) and I’m currently doing a yjs implementation with socket.io that I’d like to share via an hybrid npm package.

I would like to know if someone could help me with the necessary or recommended configuration to publish this module that I called “y-socket.io” as a hybrid package (By configuration I mean package.json, tsconfig.json and tsup.config.ts file), I already have some of this configuration implemented but I’m not sure if what I did is right.

In addition to the above, I have the following problem. I already have most of the necessary code base (server side and provider for client), since I was inspired by the y-websocket module. However, I get the error “Yjs was already imported. #438”. I already checked yjs issue #438, but I don’t quite understand how to identify the problem in my case and how to solve it.

Description of the situation:

1- I install the package locally in a project where the client-side implementation will be (because in this package I have the necessary code for the server and I also have the provider for the client), in which, in my case I use react.
2- I run “npm run build” and then “npm start” in the “y-socket.io” package to run the server, since inside the “y-socket.io” package I have an example server with express (I plan remove express afterwards, I’m just using it since it’s more comfortable for me) in the “example” folder.
3- I run the react project (or some client implementation), in which I use yjs and link a yjs document through the SocketIOProvider (provider included in “y-socket.io”)
4- When I open two tabs in the browser (to simulate more than one user connected to the same document) the error “Yjs was already imported” appears in the console. and when I try to modify the document through the different browser tabs (clients) it works in a faulty way or simply does not synchronize.
5- Considering issue #438 in the YJS repository, I have two suspicions:
5.1- I suspect that the problem is generated by the SocketIOProvider, since it is importing YJS and in the client project (react in my case) I am also importing YJS (in the same version, v13.5.41). If this were to be the problem, I do not understand how I can solve it and I would like to know if someone can guide me to solve this.
5.2- I suspect the problem may be in the package or hybrid build configuration.
5.3- The previous suspicions are also based on the fact that if I implement the SocketIOProvider in the client project without installing the package, that is, using the “provider.ts” file directly in the react project, the message “Yjs was already imported.” it doesn’t show up on console and sync works perfectly.

Considerations:

I thank you in advance for your time to read my case.

P.S.: I apologize for not having an explanation in the README.md, at the moment I don’t write the README.md completely as I don’t have client side usage examples because basically it is used the same as “y-websocket” and to run the server will simply install the package dependencies (npm install) and then run the “start” script (npm start). I honestly didn’t think it was necessary for now, due to the similarity of use it has with “y-websocket” (If you find it necessary I could create a basic one for now), but I promise a detailed README.md when I manage to solve the problem.

2

I already figured out the problem, for some strange reason when using a symlink directly in the project the problem occurs (I suspect since I’m using typescript the import is executing my package’s ts code). What I did was to build the package and use npm pack to extract the resulting zip and use that directory as a symbolic link (this way it simulated better the use of the package).

I have tested the package using commonjs with node and it works perfect if I run the server in commonjs. However, I tried to make a webpack bundle of a commonjs for the client side (using the provider) and use this bundle.js in an index.html but it didn’t work for me, so now I don’t know if my build configuration (from ts to commonsjs) is wrong, or it’s normal that it doesn’t work (I don’t know the webpack function). I would like to know if someone can solve this question for me and, if possible, guide me to adjust the configuration so that it works.

If all goes well, you should soon be able to publish the package to npm.

3

Hi @ivan-topp,

I think it’s great that you build this module!

The warning “Yjs was already imported!” often occurs if you mix cjs code with esm code. Make sure that all your files have a common extension (all ts should be fine too).

This error can be a big help. Imagine how many package you are including twice in your package just because the build manager is set up incorrectly.

It is definitely possible to bundle your package while only including Yjs once. Just sometimes our build processes are too hard to understand. Then it’s time to reduce the number of build steps (e.g. only use once bundler). In most bundlers, you can set an “alias” for packages which ensures that all references to "yjs" will import a specific file. However, that should be a last resort. (In typescript this is called an alias, terminology might differ in bundlers).

A tip form a library author: If you build a library (y-socket.io seems to be a library that can be used by others), you usually don’t want to create bundles for libraries. If y-socket.io already bundles yjs into a single file, then I will get the “Yjs is already imported!” error when I import your package and another installation of yjs. Everyone should use the same installation of Yjs (and there should only be one). Hence external packages should not be included in your bundle.

Maybe have a look at other typescript packages and see how they bundle their project.

4

Hi @dmonad and thank you for your response, for your advice and for giving me more information about the problem. For some reason, when I was testing the package through symbolic links (using npm link to create the link and then npm link y-socket.io in the project where I was going to simulate its use) I got the error “ Yjs was already imported!”. However, I later realized that I can better simulate a package install by using npm pack (to package the package) and then npm install path/to/y-socket.io.tgz. When I started testing it this way, the error stopped happening. Also mention that I did the following:

1 - I was guided by the “y-websocket” package configuration (to separate the provider files and the server part of the package)
2 - I tried using the “y-webrtc” provider in conjunction with this new provider (y-socket.io), i.e. combined as if I were combining “y-webrtc” and “y-websocket” and the synchronization worked perfectly

I would like to know if the aforementioned points make sense for you or give you more confidence about that you mentioned in your answer.

5

Ah, that makes sense. This problem is exactly why I migrated to monorepos for my
js projects.

So y-socket.io (located in your home directory) has a y-socket.io/node_modules/yjs installation and
your-other-project (also located in your home directory) also has a your-other-project/node_modules/yjs
installation. When a file in y-socket.io is executed, then it will prefer
y-socket.io/node_modules/yjs (that’s how nodejs name resolution works). Even
if you’d delete your-other-project/node_modules, it won’t be able to access
y-socket.io/node_modules/yjs because it’s not a parent-directory.

One way to get around this is to use the new subproject feature in npm. Another
approach is to implement a custom name resolution rule (e.g. an alias in
typescript) to enforce that even linked packages resolve to the desired yjs
installation.

Fyi: Your approach (packaging and installing it) works because it basically
copies everything except node_modules to
your-other-project/node_modules/y-socket.io, so it will resolve to the correct
yjs installation.

6

Aaaah, it makes a lot of sense now that you mention it, thank you very much for the clarification and for your time.

I am currently writing the README.md before publishing the package to npm.

7

Hello, I wanted to comment that I just published the package called “y-socket.io”. If anyone wants to use it feel free to do so and also make suggestions.

PS: This is my first package released on npm and it’s a way of giving back to the world in gratitude for the many packages I’ve used throughout my life as a software developer. Especially to the YJS community, since I used YJS to develop my title work.

1 Like