Repository navigation
New note and Task stop re-reading the whole vault - #73
Merged
Merged
Conversation
The Android half of this fix (ZenNotes/zennotesandroid#128, f6a9bfb there) began with New note and Quick taking 12 to 15 seconds on a 368 MB picked folder. The cause sits in code this app shares with Android line for line. app-core's refresh asks for the notes and the folders together, and each walked the whole tree on its own. A new note starts two of those refreshes at once (createAndOpen's own and the one its 'add' event starts), and returning to the app starts a rescan plus the resync it emits. The task scan read every note in the vault on every call. On Android's emulator that was 678 directory listings for one New note and 1,613 note reads for one Task. A listing here is cheaper than a picked folder's document-provider query there, but an iCloud or picked-folder vault of the same size made just as many. MobileVault's tree walks (notes, folders, assets) now share their directory listings: a walk reuses a listing another walk is fetching or fetched, while any walk runs and for one second after the last one ends, which covers the resync a rescan emits. A listing never outlives a change this vault makes: NativeFs moves a write generation before and after every mutation, and a walk that sees a new generation starts from fresh listings. That counts writes, folder creation, renames, copies and deletes, Cloud sync's verified copies, downloads and decryption, and the temp file an encrypted upload is sealed into (Android counts that one through the placeholder it writes first; here native creates the file, so the native call itself is counted). Changes from outside the app (iCloud, the Files app, another device) are found by the foreground rescan and pull to refresh, as before; a walk within that one second may miss one, the next one will not. The task scan keeps each note's parsed tasks while the note's listed mtime and size hold, the test the meta cache already uses, and drops a note's entry whenever the vault writes, moves or deletes it. What this does not do: app-core still awaits a full refresh before it opens a new note and a full task scan before "Task added". That half is in core (ZenNotes/zennotes, branch fix/create-open-before-refresh-128) and reaches this app when it adopts the next core. The vault-fs change is Android's, applied as is apart from two comments. vault-walks.test.ts is Android's test: three of its five tests fail on the old code (the inbox folder listed five times instead of once, a change from outside visible inside the window, 16 note reads instead of 11), and ignoring the write generation fails a fourth. 283 tests, typecheck and boundary check pass. The app builds for the simulator, and on an iPhone 17 simulator it creates a vault and opens the welcome note. Not yet timed on a device.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Android, New note and Quick took 12 to 15 seconds on a large picked folder (ZenNotes/zennotesandroid#128). The cause is in bridge code this app shares with Android line for line: app-core's refresh walks the whole vault once for notes and again for folders, a new note starts two refreshes at once, a return to the app starts a rescan plus its resync, and every task scan re-reads every note. On Android's emulator that was 678 directory listings per New note and 1,613 note reads per Task. This is the same shell fix, ported.
77612a9fix(mobile):MobileVault's note, folder and asset walks share directory listings while any walk runs and for one second after the last ends.NativeFsmoves awriteGenerationbefore and after every mutation (writes, mkdir, rename, copy, delete, Cloud sync's copies, downloads and decryption, and the encrypted upload's temp file), and a walk that sees a new generation lists afresh, so a listing never hides a change the app made. Outside changes (iCloud, the Files app, another device) are found by the foreground rescan and pull to refresh, as before. The task scan reuses a note's parsed tasks while its listed mtime and size hold, and drops the entry whenever the vault writes, moves or deletes the note.The
vault-fs.tschange is Android's, applied as is apart from two comments;vault-walks.test.tsis Android's test. Three of its five tests fail on the old code (the inbox listed five times instead of once, an outside change visible inside the window, 16 note reads instead of 11), and ignoring the write generation fails a fourth. 283 tests, typecheck and boundary check pass. The app builds for the simulator and on an iPhone 17 simulator creates a vault and opens the welcome note. Not yet timed on a device.Not in this PR: app-core still awaits a full refresh before opening a new note and a full task scan before "Task added". That half is in core (ZenNotes/zennotes, branch
fix/create-open-before-refresh-128) and arrives with the next core adoption.How to test locally:
All five pass. To see the old behavior,
git checkout origin/main -- src/bridge/vault-fs.ts src/bridge/native-fs.tsand run it again: three fail. Restore withgit checkout HEAD -- src/bridge. In the app:npm run sync(CocoaPods needsLANG=en_US.UTF-8), run on a simulator from Xcode, then create a note, add a Task and check a task off: each change shows at once in the note list and in Tasks, which is the write generation and the task cache doing their job.Android counterpart: f6a9bfb on ZenNotes/zennotesandroid branch
fix/create-note-perf-128(no PR yet).