Caught error while handling a Yjs update RangeError

Hey folks!

I am facing an issue with persisting my data on my Ydoc which is largely driven by this error:

sync.js:86 Caught error while handling a Yjs update RangeError: Maximum call stack size exceeded
at Transaction.js:307:38
at Map.forEach ()
at cleanupTransactions (Transaction.js:307:30)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
at cleanupTransactions (Transaction.js:370:9)
readSyncStep2 @ sync.js:86
readSyncMessage @ sync.js:121
applySyncMessage @ MessageReceiver.ts:70
apply @ MessageReceiver.ts:31
onMessage @ HocuspocusProvider.ts:561

Can some one help debug this? any recommendations?

Does this happen on server-side or client-side? Probably TipTap discord is your best bet finding help. I’m 99% sure it’s a problem with Hocuspocus/TipTap yjs integration.

I believe its a server side error - I have tested the code with the backend switched off and the original hocuspocus plug-ins and then the error does not seem to appear; it appears when i use the supabase postgres database - so from this i am infering it is a server side error

This is the crux of the code:

using pg client

const server = Server.configure({
port: 8080,
extensions: [
new Database({
// Return a Promise to retrieve data …
fetch: async ({ documentName }) => {

    return new Promise((resolve, reject) => {
      const text = 'SELECT data FROM "documents" WHERE name = $1';
      const values = [documentName];
      console.log("documentname");
      console.log(documentName);

      client.query(text, values, (err, res) => {
        if (err) {
          console.log(err);
          reject(err);
        } else {
          // console.log(res.rows[0]);
          console.log("buffer code")
          console.log(Buffer.from(res.rows[0].data.data));
          const YDOC = new Y.Doc();
          Y.applyUpdate(YDOC, Buffer.from(res.rows[0].data.data));
          console.log([...Buffer.from(res.rows[0].data.data)]);
          resolve(Buffer.from(res.rows[0].data.data));
        }
      });
    });
  },
  // … and a Promise to store data:
  store: async ({ documentName, state }) => {
    const text = `INSERT INTO "documents" ("name", "data") VALUES ($1, $2)
        ON CONFLICT(name) DO UPDATE SET data = $2`;
    console.log('state', state);
    const values = [documentName, JSON.stringify(state)];
    const res = await client.query(text, values);
    console.log('store log', res);
  },
}),

],
});

I’ll post it in discord as well.

Well you definitely need to choose whether you store the doc in JSON or in binary. Now I’m not sure which one you are using - on the other hand you’re inserting JSON with JSON.stringify(state) but then hydrating using a binary: Y.applyUpdate(YDOC, Buffer.from(res.rows[0].data.data)).

Remember to convert it back to binary with prosemirrorToYDoc and encodeStateAsUpdate (or alternatively just replace the Ydoc from the one fetched from Postgres). Or store the doc in binary in the first place.

Thanks for the reply. I will give some context about how the data is being stored, and this should shed some light on why I am using the JSON.stringify() method:

I have created a table called “documents” in Supabase. It has 3 columns: name, date and data. The data column is of jsob data type. Supabase database has jsonb data type for binary type data. The data column stores data in the following format: {“data”:[0,0], “type”:”Buffer”}.

Now when I update the database, I cannot use the state variable directly because the state variable contains data in Buffer type format. To insert this into Supabase, I have to apply JSON.stringify. For example, this makes the <Buffer 00 00>, into {“data”:[0,0], “type”:”Buffer”}, which is the manner in which I am storing data in the Supabase column. I think the data value is still of binary data type.

One question arises, that I am doing Y.applyUpdate(YDOC, res.rows[0].data.data); and using resolve(Buffer.from(res.rows[0].data.data)).

Can I also do: Y.applyUpdate(YDOC, Buffer.from(res.rows[0].data.data)) ? What is the difference between these two approaches ? Because there is not much difference between the array being stored in the Supabase (which is Unit8Array format from my current understanding) and the Buffer NodeJS format.

This is my thought process; appreciate your feedback and help on the matter.

Funny enough, I received the same RangeError and lo and behold, googling brought me to this same thread I previously commented on :sweat_smile:.

Poking around my code, it proved to be quite tedious bug indeed but eventually I realized it was one of the those classic Yjs bugs. So when you load your editor, you must hold off from rendering it until the doc has synced. At least this has been my experience — otherwise you’ll get bugs like these where Yjs can’t transform the empty, default doc (single paragraph) into whatever it receives from the provider.

It’s very confusing but so this is what I do:

  const [synced, setSynced] = useState(false)
  // Must be ref since in React.StrictMode effects are called twice _before_ useState hooks have chance to update.......
  const loading = useRef<Loading | undefined>()
  const [initialData, setInitialData] = useState<InitialData | undefined>(undefined)

  useEffect(() => {
    const cur = loading.current
    if (
      sessionUser &&
      syncToken &&
      cur?.docId !== params.docId &&
      notEqualUser(cur?.user, sessionUser) &&
      cur?.token !== syncToken
    ) {
      initializeEditor(params.docId, sessionUser, syncToken)
    }
    setCurrentDocId(params.docId)
    return () => {
      // @TODO this has to be destroyed but since useEffect hooks are so flaky and useless, it has to be done
      // by listening to navigation hooks or something, maybe in editor.tsx
      // loading.current?.provider.destroy()
      setCurrentDocId()
    }
  }, [params.docId, sessionUser, syncToken])

  function initializeEditor(docId: string, user: User & { id: string }, token: string) {
    setSynced(false)
    loading.current?.provider.destroy()
    const ydoc = new Y.Doc()
    const provider = new HocuspocusProvider({
      url: client.HOCUSPOCUS_URL,
      name: docId,
      document: ydoc,
      token,
      onSynced: () => {
        setSynced(true)
      }
    })
    loading.current = {
      docId,
      provider,
      token,
      user
    }
    setInitialData({
      docId,
      ydoc,
      provider,
      token,
      user: {
        id: user.id,
        name: user.name || 'Unknown',
        image: user.image || null
      }
    })
    setSyncToken(token)
  }

and in render:

{initialData && synced && <TiptapEditor initialData={initialData} />}

Note the hack for double useEffect as it’s otherwise quite annoying when developing. But the important part is that the Tiptap editor receives the initial data only when the provider has fully synced. In other version I don’t destroy the editor in-between changes — just keep the old one until the new one has loaded — but it’s a lot easier this way.

As an update, I think the exact reason for this error is that I have non-standard schema which doesn’t use basic doc > p as default docs. When I render the editor without provider being synced, the empty Y.Doc — single paragraph — automatically updates itself to the custom schema. That wouldn’t be an issue except when the synced doc is finally received from the server it so happens — as conflict-free data are designed — it joins with the received doc.

So suddenly there’s two document roots which is incompatible with the schema and thus the other part is thrown out by the ProseMirror editor and Yjs proceeds to explode as the binding is somehow corrupted. Or something like that.

It would be nice to have better error handling incase this happens or some type of defaultSchema as parameter to Y.XmlFragment (for the Y.Doc). I suppose you could write one yourself. Oh well.