How to sync thousands of documents and have local persistent store?

An append-only file on disk would work fine. In order to reduce metadata, you might want to concatenate the updates from time to time. Either by calling Y.encodeStateAsUpdate or by using the new differential updates feature (not yet released).

Why don’t you start with y-leveldb? It works very well in nodejs. Alternatively you can use y-indexeddb, which is supported in the browser and in electron.
There is a separate section about offline editing in the docs: Offline Support | Yjs Docs & y-indexeddb | Yjs Docs

I built the subdocuments feature exactly for this. You can manage all your Yjs documents as references from a top-level Yjs document. The provider would be responsible for syncing subdocuments efficiently.

At the moment, none of the official providers support efficient syncing subdocuments. I recommend to adapt an existing provider and implement a syncing mechanism that makes sense for your application. For example, you could maintain a “last-modified” field that is updated when other clients should sync the document.