Doc TypeError in Jest test environment

Hello everyone,

I am trying to test yjs with Jest. Unfortunately I was not able to get an initial test running because of a TypeError.
Building my code and running it in the browser works. I was not able to get the yjs Doc running in the test environment.
(Yes, I could mock yjs, but in this case I want to test my code together with the yjs library)

Here is a basic dummy example I was not able to run:

// example.spec.ts

import { Doc } from "yjs";

describe("example test", () => {

  it("should be defined", () => {
    const doc = new Doc();

    expect(doc).toBeDefined();
  });

});

Error message:

TypeError: _yjs.Doc is not a constructor

  4 | 
  5 |   it("should be defined", () => {
> 6 |     const doc = new Doc();
    |                 ^

The error message when trying to import the doc from the dist directory:

  ā— Test suite failed to run

    Cannot find module 'yjs/dist/src/internals' from 'example.spec.ts'

    > 1 | import { Doc } from "yjs/dist/src/internals";
        | ^
      2 | 

Does anyone have an idea, how to approach or solve this error?
(Sry for my beginner question :see_no_evil:)

Hi @flow,

Iā€™m sorry this is not working for you. Iā€™m partially to blame for this because I export Yjs as a ESM module (a fairly new standardized javascript module format). A lot of old bundlers donā€™t interpret ESM modules properly and then you get errors like yours. I still export Yjs as a ESM module because it reduces the bundle size and this module format works natively in the browser.

Some things you could try:

  • Import the cjs module (the old module format that all bundlers understand). Try import * as Y from "yjs/dist/yjs.cjs"
  • Make sure to use the import * as Y syntax, and not import { Doc } from 'yjs'. A lot of old bundlers differentiate between these two (especially if they depend on the cjs bundle).
  • Import the mjs module explicitly (the new module format). Try import * as Y from "yjs/dist/yjs.mjs"

Let me know if any of the proposed solutions works. If not, we will find another solution.

Hi @dmonad,

thank you for the fast answer, hints with the modules and the proposed solutions!
Unfortunately I was not able to get my setup working. So I ran some more tests.

Here is my setup and the things I tried that did or did not work out:

Setup

I want to use yjs in a react application. Moreover, I use create-react-app which offers a maintained build and test environment.
I would say that create-react-app is the most common tool for building react applicaions. Everything is maintained in the react-scripts package. The configuration options for jest are quite limited, so extending their config might not be ideal. There is an option to eject the config but then you loose support by the tool. So it would be great to find a solution without ejecting.

Problem

For everyone new to the topic, here is a post, I would say describes the problem quite well:

The following issue describes the problem with .mjs files, CRA and Jest:

Might there be a way to support older bundlers with yjs or do we have to wait until Jest officially supports it and create-react-app adapts this version of jest?

My experiments

I created a demo project for trying out the different import statements. Unfortunately none of them worked. The problem is when importing the .mjs or .cjs file directly, the runner (and also my IDE type checking) was not able to find the files.

Commit with the test files:

The GitHub Action results from running the tests:


Another approach was to clone the yjs repo and adapt the package.json & rollup.config.js to the following (only showing my changes):

// rollup.config.js:

...previos imports

import pkg from "./package.json"; // <------ added

export default [{
  input: './src/index.js',
  output: {
    name: 'Y',
    file: pkg.main,  // <---------------- changed
    format: 'cjs',
    sourcemap: true,
    ...
  },
  external: id => /^lib0\//.test(id)
}, {
  input: './src/index.js',
  output: {
    name: 'Y',
    file: pkg.module,   // <---------------- changed
    format: 'esm',
    sourcemap: true
  },
  external: id => /^lib0\//.test(id)

package.json

{
    "main": "./dist/yjs.js",
    "module": "./dist/yjs.esm.js",
    "unpkg": "./dist/yjs.esm.js",
}

This brought me that step further, that it could find the Doc. But the next error was that it does not find the Observerable class from lib0.

I stopped my experimenting here, because I am not sure, if this is the right way to approach the problem. As far as I get it, yjs bundels with the new javascript module format (or as some might call it ā€œMichael Jackson Scriptsā€ :wink: ). Webpack 4, Babel or in this case here Jest lack behind which makes importing .mjs files quite hard / not possible for now.
So might there be an option in yjs bundling to support both options?


Some more things that I encountered:

It is quite interesting, that when I start a codesandbox (default react typescript template) with react, react-scripts and typescript (which uses an older version of all the packages and node 10.x instead of 14.x) the test works with the ESM module import.
Here is the setup (on top the right side browser preview window there is a Tab which states ā€œTestsā€ where you can open the test results):


At the moment I am a little confused, why it seems to work in CodeSandbox but not in a new local setup of create-react-app.

Any ideas for new approaches or possible solutions?

Hi @flow seems you really took a stab at js modules!

This is a really nice trick :+1: Makes it foolproof to generate the correct files. But this is basically what is already happening. Yjs exports both a cjs and a esm module. The main bundle links to dist/yjs.cjs which is a commonjs bundle. You can do const Y = require('yjs') / import * as Y from 'yjs' in nodejs, webpack, and rollup.

Thanks for creating a repository that I can debug. It seems that Jest is simply importing a string instead of the javascript bundle. It doesnā€™t correctly handle files that end with .cjs - although the modules working group recommends to name files like this. When you rename all modules to .js files then it will work as expected (also the lib0 & isomorpic.js modules).

Screenshot from 2020-10-28 14-48-01

So, on the one hand, we have recommendations from the modules working group, on the other hand we have esoteric bundlers that do magical things to javascript packages. It is truly frustrating that every bundler is doing its own thing. Iā€™m sure there is a solution to make Yjs work with Jest, but I rather want to follow recommendations to make Yjs work natively in nodejs.

I made the choice to make Yjs a native esm module, and keep cjs compatibility as far as possible. I want to push a better module standard and make Yjs work without a bundling step (yes, this is already possible). I favor to make Yjs work over unpkg and work in the browser without a bundler (the modules working group are working on a path-mapper that will make this possible). If this means to break some esoteric bundlers Iā€™m fine with it.

Sorry @flow , this is probably not the answer you were hoping for. While researching this problem I stumbled across some people who use babel to transform modules beforehand. Maybe there is a way to support *.cjs files in jest. You could try using babel or create-react-app-ts. It probably doesnā€™t hurt to open a ticket in the repository of the module bundler (it is not clear to me what they are usingā€¦ is react-scripts using babel, webpack, or is it a custom bundler?).

1 Like

Hi @dmonad,

big thank you for further investigation!

To be honest I expected an similar answer and I totally understand and share your opinion.

Sure, I hoped there would be some sort of temporary workaround.
It is quite a bummer that it does not work work with a default react setup, because of the legacy bundlers.
If I find a temporary workaround until the those problems are solved, I will give an update in this thread.


Regarding react-scripts, they use webpack and babel. You could run npm run eject to have a complete view at the build process.


Thank you again, for your help!

That would be great, thanks!

Pure webpack can include Yjs without a problem. I donā€™t understand why the yjs.cjs content is replaced by a string, this seems like a bug.

@flow did you find a temporary solution in the end ?
Thanks !

Hey, @flow @dmonad ,
I just encountered the same exact issue, did you find any solution for it? couldnā€™t solve it using the cjs/mjs modules

Iā€™ve found a way to solve the problem

create a new file with this content and import it in the test

module.exports = {
     ...jest.requireActual('yjs')
};

FYI Here is how I got yjs working in an old jest + create-react-app + typescript testing environment. I didnā€™t have to change my import syntax once the babel transformation was fixed.

Error 1:

SyntaxError: Cannot use import statement outside a module

      at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1350:14)
      at Object.<anonymous> (node_modules/yjs/dist/yjs.cjs:3:18)

To solve this, I modified the transformIgnorePatterns jest config to not ignore yjs, y-indexeddb, and lib0. They need to be transformed by babel since they are esm modules:

react-scripts test --transformIgnorePatterns "node_modules/(?\!yjs|y-indexeddb|lib0)"

Error 2:

ReferenceError: crypto is not defined

To solve the this, I defined global.crypto in setupTests.js which jest loads before all tests:

import crypto from 'crypto'

Object.defineProperty(global, 'crypto', {
  value: {
    getRandomValues: arr => crypto.randomBytes(arr.length),
  },
})

References:

1 Like